I'm a little lost here.  if he has HSV1 and 2 - he can have it without rash recurrences - which makes it more difficult to know when he can pass the virus to you.  If your IgG and IgM tests are negative then I would practice safe sex with a condom ALWAYS to raise your chances of not getting herpes.  You can't depend on using the rash as an indication (his rash) of whether he is infectious - it's not a good guideline -  so you need to practice safe sex all the time.   If your IgG and IgM are negative then I would see a gynecologist to find out why you are suffering with this ongoing labial pain.  You need to be examined, cultured and if necessary have a work up - if the diagnosis is not obvious on examination.