Thanks everyone for all the kind words... I really appreciate all the support here!
For anyone thinking about doing the same thing as me feel free to message me with any questions. I just got a PM from someone and as part of my long response I suggest the following to anyone thinking about doing the same thing...
What I would suggest to you is this:
1) Go out and buy
The 4 Hour Work Week... it's a great book and will motivate you A LOT! I read it and it really helped me.
2) Read
Jeremy's blog - He has a ton of great motivational posts around this subject.
3) Think of the WORSE case scenario and weigh it to the BEST case scenario... if the risk is minimal, but the potential for reward is great, I think you already know what to do.
4) Think what the best possible thing that could happen at your current job... if you think that would make you way happier than doing affiliate stuff / your own thing, then maybe you actually do like your job.
5) Do what will make YOU happy... if your current job is killing you (I know mine was), its only going to get worse and worse.
6) Talk to your family and friends - I spoke to my parents about it before I made my decision. I thought they would give me a hard time as I was always raised to go to school, get the good job etc. and work for a company, but the reaction I got was quite the opposite. They were extremely supportive which made the decision so much easier!
Good luck!