Although I have no idea how to answer your question, I can say that as a little girl I was madly, passionately in love with Sherlock Holmes. So much so, that I read every book and story, and dreamed about him at night. Then, in my teens, I feel in love with Mr. Rochester, only to be shortly replaced by Mr. Darcy. In college, it was Prince Andrei from War and Peace. And now, my heart belongs (well, metaphorically) to Robert Plant. At least, unlike the others, Plant is not a fictional creation.
But to get back on point a bit, I think Doyle had to make Scotland yard look bad in order to present Sherlock Holmes as a master genius. It was entertaining fiction, and I believe people took it as such. I'm sure there are historical works out there describing the operations of Scotland yard during the time period in question for those who are truly curious. (A trip to the library or even a knowledgeable forum member might provide the answer.) But somehow, I doubt Scotland Yard was any worse than U.S. Police today (without considering the tremendous improvements in GPS tracking, forensics, and etc.).
The bottom line, however, is that without a master detective, there is no detective story.