Hello Danny,
I first wanted to tell you (you probably already know)that the coupling between TOMAWAC and TELEMAC3D is actually a 2D coupling : the wave driving forces FX and FY acting on the current are uniformly imposed over the water depth.
One question : did you try to run your case in 2D?
We are actually working on the improvement of this coupling but it won't be available before at least one more year. The PhD student who works on improving this method already had some problems with the wave set-down (so I am not surprised you also had problems with it) and is actually trying to solve them.
Actually, the only way to affect the wave set-down in the current version would be to modify the wave breaking parameters in TOMAWAC. Nevertheless, this would also have an impact on your wave heights and therefore you will have to be cautious and check the wave heights are still good.
Best regards,
Elodie