The whole situation with Drewes, Edith, Marigold is a big mess and multifaceted, and i tried to put it all in a pattern that makes sense to me:
Drewes helped Crawley family with the farm and Edith with Marigold. They lost the child and the farm to no fault of their own. Certainly not fair.
Sir Robert’s priority is Edith’s well-being. I think he could have done a little more in favor of Drewes to balance out this situation.
A sympathetic character, Edith, poor little Edith, suddenly is not so benign, and bluntly selfish, when her interests are at stake, but then can one blame a Mother who had her child under such circumstances. First she panicked, made one set of choices, now she changed her mind, it affect others negatively.
Perhaps Lady Mary had a point smirking at Edith from time to time, perhaps since child-hood she knew her better than audience.
The only “minus” against Mrs. Drewe that could justify Crawley’s decisions, is her obsession with Marigold. If she was childless and Marigold was given to her as her only child, it would be understandable. But she has many children, and did not give birth to Marigold, and did not raise her for many years, but rather few months. She should not have taken Marigold off the show-grounds. That sealed the deal. She basically kidnapped the child, and now there is no excuse.
Edith, on the other hand, should have been more flexible in letting Drewes “share custody” of Marigold. At least TRY to let the child spend some time with them. It is possible that Mrs. Drewe might have done something unacceptable in such situation, but at least they should have TRIED.
Sir Robert should have influenced Edith to make such attempts. Mary should not have taken Marigold to the farm grounds, I am not even sure she had “noble motivations” to do so. But it seems she does not approve of Edith keeping Marigold completely away from Drewes. I am not sure if she “created” an additional problem, or placed a final straw on top of a situation which eventually would have exploded anyway.
Asking Drewes to leave the farm is too drastic, imho. Asking Mr. Drewe to keep Mrs. Drewe away from Marigold, regardless of how Edith feels about their proximity to the child, would have been the right things to do.
The whole situation looks like “rich took advantage of the less fortunate”, got the help, then took the child and ended their residency. However, if such case of adoptive vs. biological parent came to court, given the child was with adoptive parents just few months, and birth parent had the means to support the child, the court would have awarded Marigold to Edith.
There are many cases, when a wealthy family adopts a child of a poor single mother, who later changes her mind, and court awards the baby to the Mother.
So the situation is not completely unfair. All I can say about "all parties involved"...... they are all human, and ALL made mistakes..
The next county will probably suffice. 
