While cognitive impairments are prevalent in first-episode psychosis, the course of these deficits is not fully understood. Most deficits appear to remain stable, however there is uncertainty regarding the trajectory of specific cognitive domains after illness onset. This study investigates the longitudinal course of cognitive deficits four years after a first-episode of psychosis and the relationship of performance with clinical course and response to treatment. Twenty three individuals with psychotic illness, matched with 21 healthy volunteers, were assessed using the MATRICS Consensus Cognitive Battery at illness onset and 4 years later. We also investigated the relationship between cognitive deficits and quality of life and clinical indices. Verbal learning and two measures of processing speed had marked poorer trajectory over four years compared to the remaining cognitive domains. Processing speed performance was found to contribute to the cognitive deficits in psychosis. Poorer clinical outcome was associated with greater deficits at illness onset in reasoning and problem solving and social cognition. Cognitive deficits did not predict quality of life at follow-up, nor did diagnosis subtype differentiate cognitive performance. In conclusion, an initial psychotic episode may be associated with an additional cost on verbal learning and two measures of processing speed over a time spanning at least four years. Moreover, processing speed, which has been manipulated through intervention in previous studies, may represent a viable therapeutic target. Finally, cognition at illness onset may have a predictive capability of illness course.