Thanks for sending it to me, c2i.
I would have to say that the study is incomplete, at best. It takes test scores from only 4th and 8th grade students in one school year, 2005-2006. The districts studied had to have more than 10,000 students. That means that the scores are automatically skewed towards urban areas, which means higher poverty rates. The test scores are from a site that they acknowledge, "subsequently disappeared from the internet, presumably because it was not profitable." They also only used the "state-wide membership dues for the largest union in the state," which in most cases will be the state chapter of the NEA or AFT. The dues for those two unions will be higher than any local union, so their "union dues/teacher" numbers could be higher than the true amounts. It seems, if I'm reading it all correctly, that they tried to make the union dues portion as high as they could and the test scores as low as they could in the way they selected their data.
It's not surprising that they found the biggest districts to have lower test scores, and I'd guess if you checked poverty rates in those districts you'd find a correlation there.
One of the authors of the study, Johnathon Lott, is quite right-leaning as well, so I'm not surprised at how the results panned out.