This is a question I've been avoiding addressing, simply because I'm not a Dr, and even if I was, I'd need to look at your patient history and talk with you before reaching a conclusion. So if you've clicked onto this post looking for a clear-cut answer, you've drawn the short straw. Go to your GP!
Instead, I'm going to talk a little bit about my experience. Throughout 6 solid years of depression and anxiety, I tried medication a grand total of once on the advice of a psychiatrist who noticed I couldn't hold a conversation without panicking and becoming breathless. I've told this story before, so for the sake of not boring my long-term readers, two weeks of hellish fatigue culminated in me stopping the medication immediately without consulting any healthcare professionals (I do not recommend this).
Since then I have refused point blank to try any other medications. I was determined I was going to get through it all with sheer willpower - again, not the best attitude to have. I probably could have recovered a lot quicker had I taken the time to find a medication that was right for me. Instead I spent another three years fighting what felt like a losing battle. Eventually I found a therapist that I clicked with and made a few lifestyle changes.
Now here I am. Things aren't perfect, but for the first time in 6 years I can say that I am depression free, and I can hold a conversation on a good day - probably most significantly, I now have more good days than bad days.
I think my attitude towards medication has changed a lot because of my resistance towards anti-depressants. My attitude seems to be that if I can get through 6 years of anxiety and depression without relying on medication, then I can grit my teeth and work my way through pain rather than taking painkillers. I can work through the night and sleep through the day rather than taking sleeping tablets or muscle relaxants. I can get through anything without using medication as a crutch.
I've yet to decide whether or not it's a good thing that I'm voluntarily reducing my quality of life in order to avoid taking medication. I suspect it's probably not so great. Still...one battle at a time.