Newsflash: Builder developing on neighboring property

So we bought a new construction house in 2/2021 didn’t close until 6/2021 bc the builder had construction liens they had to clear up..they bought our lot plus the neighboring lot for dirt cheap and had planned on making twin houses sharing a driveway and halfway into construction they learned the city does not allow shared driveways. We had thought their other lot was essentially unbuildable because it has very little street frontage and has a huge slope as well as constantly floods…welllll we just learned the same builder (that we are about to sue for breach of contract having never finished many things they were supposed to) is building a spec house on the lot next to us. We have raised questions about it fitting and maintaining setbacks and they assure everyone this 5br house will fit and they will be able to maintain the setback requirements. Long story short, is there anything we can do to stop them? We offered to buy the lit they asked for 40k (they paid 4k for the lot our house is on and the other in 2018). 40k is ridiculous but we really don’t want a neighbor that close. We had asked our realtor to include a right of first refusal in our contract and somehow it got left out of the final and we didn’t notice. we don’t know how they plan to build considering it is difficult to even access their property without using our driveway (and we have made it clear that they can’t) but is there anything else we can do? These guys are known in our neighborhood for taking foreverrr and not doing anything correctly. They change their business names a lot to avoid lawsuits etc. Is what the HOA director told us. The HOA can only do so much as far as only allowing them to use certain colors and finishes etc. The director is our neighbor on the other side and he is also against them building but not sure how to legally stop them. Ideas??