I'm not sure what you mean by harder, because I find things about the same, if not easier(thanks internet), after leaving school and understanding how our memory works.
I think you're noticing the difference between a teacher and a book. A teacher will draw pictures, use gestures, tone of voice, and answer any questions. A book will just tell you things, and assume you understand, and leave you to set the tone of voice.
If you want to learn maths, give the site Khan Academy a go. Also if you really want to learn, use a flashcard program like Anki. I use Anki for learning everything. Once you learn something, you easily forget it. I use Anki to keep reminding me of whatever I'm learning, so I never forget it. You do have to use it every day, but it's a good habit to get into, whilst having breakfast.
EDIT: added this:
You'll also notice on Khan Academy, Sal Khan uses just a blackboard and his voice, because the tone of voice really helps get the point across, also he draws and moves things around, so you can see and hear how it all goes together.