can the buyer pay the down payment directly to the seller?
Yes you could, however it is not recommended. About the only time I would consider it is if there are no Realtors involved and even then I do not believe it is a good practice. In the event any issues arise it may be difficult to obtain a refund if the transaction is not consumated without taking legal action...