First, review answer posted to the ARCHCareers blog - http://archcareers.blogspot.
With your undergraduate degree in mathematics, you are eligible to apply to the Master of Architecture (3-4 years). There are about 50 programs or so. Begin to research programs via the following websites --
While researching programs, discover what prerequisites each program may require. Most require calculus; some will require physics and others may require freehand drawing and architectural history. To the extent possible, attempt to meet these during your senior year especially freehand drawing as you will need to submit a portfolio to each graduate program.
If possible, consider visiting some programs to learn about them but also to talk with current students. Looking at a map, the closest program may be Syracuse University.