Please know that I am not a doctor or a dermatologist. Your guess could be right. The fancy name for an infected hair follicle is folliculitis.
Medline Plus has this advice about treating folliculitis:
"Hot, moist compresses may promote drainage of the affected follicles. Treatment may include antibiotics applied to the skin (mupirocin) or taken by mouth (dicloxacillin), or antifungal medications to control the infection."
It may be this and it may be something else entirely. You won't know until you get it seen by a doctor.
Hope you feel better soon. Thank you for your question.