Yes, this would be a water mold problem. Lavender wants a Mediterranean situation, with hot dry summers, warm damp winters and excellent drainage. The dieback problem is persvasive enough that at least a few cultivars marketed as being disease resistant have been presented in the past.

The clay soil is the kiss of death for them...If you have clay soil and want lavender, make mounds of a gritty, fast-draining soil and plant them in those. Lavender is short-lived anyway (5-7 years - after that they get very woody and tatty looking) so you don't have to worry about them 'outgrowing' their mounds. Works here very well.

Matt,
I see a drip hose in the pic. Right above the lavender that are rotten.
No drip hose on lavendar.
Lavender do not tolerate wet and heat at the same time.
THe only time you can water them is at night, when it cools off.
THe drip hose is watering them in the heat.
I don't water my lavender at all, ever, so the only water they get is when it rains, and it cools off then.
If you really want to grow them in clay, the above poster had it right. Lots of drainage.
i grow mine on a sandhill.
Literally.
Easiest plant I have, I do absolutely nothing to it.
LOL!