College Access and Affordability Act

College Access and Affordability Act - Amends the Internal Revenue Code to: (1) increase to $4,000 the maximum amount of qualified tuition and related expenses eligible for the Hope Scholarship Credit and adjust such amount for inflation beginning in 2006; (2) increase to $58,000 the modified adjusted gross income threshold for determining reductions in the allowable amount of the Hope Scholarship Credit; and (3) increase from two to four the number of years a Hope Scholarship Credit may be claimed.

Amends the Higher Education Act of 1965 to provide for student loan forgiveness up to $20,000 for individuals in qualified public service positions who are not in default of their loan obligations. Defines "qualified public service positions" to include: (1) highly qualified teachers of mathematics, science, and bilingual and special education in low-income communities; (2) firefighters, police officers, or emergency medical technicians serving low-income communities; (3) nurses serving low-income communities; and (4) public or private child welfare workers.