Collection of functional data has become more prevalent in the past decade, including functional data collected longitudinally. For example, in the HIV Neuroimaging Consortium (HIVNC) study, magnetic resonance spectroscopy (MRS) was used to collect metabolite spectra from multiple brain regions at a number of time points. Analysis of such data usually follows a two-step procedure: (1) metabolite concentration extraction and (2) association study of extracted covariates and outcomes of interest. Our approach does not rely on the frequently unreliable feature extraction. Instead, it uses scientific knowledge to estimate regression function without explicitly extracting the feature characteristics. Specifically, we propose a method for functional linear model estimation using partially empirical eigenvectors for regression (PEER) in the longitudinal data setting. Our method allows the regression function to vary across both time and space. We derive the estimator's statistical properties and discuss their connections with the generalized singular value decomposition (GSVD). The results of the simulation studies and an application to the analysis of HIV patients' neurocognitive impairment as a function of MRS data are presented.