Home furnishings really don't have to cost a lot of money, low cost furniture can be found in thrift stores all the time.
I found an advertisement for a free high back white chair on Gumtree, the Australian classified ad website. I went to collect the chair, which happened to be from an op shop (Australian thrift store) sorting facility. The chair couldn't be sold as it was, because of a stain on the seat cushion.
I took the chair home, and found a suitable fabric from my thrifted fabric collection to re-cover the seat cushion and give it a new lease of life.
I'm by no means an expert at upholstery, a complete novice actually, but managed to recover the cushion using a hot glue gun to the underside of the seat. I intend to reinforce the new fabric cover with a staple gun at some point.
A very simple change, and yet a big difference to the chair.