Michael Schwandt has written, communicated, and thought extensively about quality management on SDN, discussion forums, wiki pages and blog posts. Here are his recent posts:
- Quality: Invitation To Clean Up Your Forum Threads
- Update on Quality, Control and Usability
- Improve Quality and Usability in Forums
- Next steps on improving quality
- Moving to Strict Rules to improve Quality and Control
To see how things are going, I took a look around this morning.
Forums
I dug into 2 forums, with over 300,000 messages and 60,000 threads each. From a summary page, I drilled into posters to see what is available (how sharing/caring are they), and how many posts and unanswered questions they have (how neat/tidy). I feel qualified to do this is as I’ve closed my open questions. I’ve omitted personal data.
Forum 1
- 3 out of 14 posters had an email address on their forum profile
- 3 out of 14 (but not the same 3) had email, cell number or other on their business card
- 4 posters had more than 10 open questions (1 had over 100)
- There were 15 posts, but only 14 unique posters in this sample
Forum 2
- 4 out of 14 posters had an email address on their forum profile
- 2 out of 14 (none of the prior 4) had email, cell number or other on their business card
- 4 posters had more than 10 open questions (2 had over 100, one astoundingly close to 400)
- Likewise, 1 duplicate poster
Results
| Forum | Posters | Total Posts | Open Questions |
|---|---|---|---|
| 1 | 14 | 313 | 222 |
| 2 | 14 | 1,442 | 754 |
I’d say the “you can’t ask new questions if you have more than 10 open” rule isn’t working.
There is also a blatantly anonymous user id that I’ll not chase today.
Wiki
I tried to see wiki traffic patterns, and was side-tracked by quality issues. I am leery of page ID numbers like 68,747,521 being actual counts, as over a decade of SAP notes by thousands of authors topped 1 milllion not long ago.
As Dave Barry always said, “I am not making this up!” — I picked a recently edited wiki page to view number of edits, duration, and found:
Instead of the advertised content, it’s a flaming, misspelled, low (or no) quality discussion forum post. I’ve spray-painted the user ID on the image; it’s not hard to find. If the author meant to improve wiki quality, they missed seriously.
Above is the slightly-hard-to-find RESTORE button I used to revert the last good version of this wiki page.
Moral
This isn’t a moral of the above content, it’s a return to the New Year’s Resolution theme. My company has a matching gift program where my donations to charities are met 100% up to an annual limit. I just looked back at the last 4 years and found I haven’t hit that limit. I intend to go beyond in 2009.
2008-12-26 22:43:30 Manish Meshram Business Card [Reply]
I cannot find J2EE engine on SAP portal for download Plz any one send link for downloading J2EE engine ".
I am on vacation, but have to keep my blackberry on, since I am waiting for an important email. It buzzed at some unearthly hour saying that some one commented on my blog. I scrolled on (still don't know why I would do that ) to find the request for J2EE material, shouted something that is not polite, and ended up waking up my family in the process. This has become a quality of life issue 🙂
It was implemented on the 1st of July 2008, and only questions opened subsequently are included in the counter. Previous questions are not affected - although some people also misunderstand this and close all their old questions as well (unfortunately with the occasional "asdf" comments (but these are deleted again).
There is nothing wrong with doing what you did, but it is no valid reason to rant about the 10 open question rule 😉
Cheers,
Julius
Jim
Of course closing earlier ones which were neglected is a good deed as well (I also had a few), but this was intentionally not the motive behind the 10 open question rules as far as I know (otherwise it could easily have been implemented).
What would be interesting to compare (in your analysis) is the behaviour of the folks with open questions since July 2008, to their behaviour before that date.
That has in my opinion improved in the forums I moderate. Of course there still are strategists who double post and use 2 accounts etc - they will never go away completely and are often the main cause behind neglected open threads. We try to catch them early.
Cheers,
Julius
Michael Schwandt Quality: Invitation To Clean Up Your Forum Threads: "We want to start an initiative to clean up our threads! Let's clean up any thread which is not answered properly or contains comments, which indicate that it has a repetitive topic...
Ideally you are able to reduce the unresolved questions to zero."
His goal is clear to me.
As for further analysis, I'll leave that to those with better tools, more time, or both.
Jim
Some are just bone dry lazy anway... in which case I post an extract from their profile showing the unresolved thread number and sometimes a link to other ID's they use as well.
But your statement that the 10 open question rule does not work is not correct. And your quote from Michael Schwandt's blog relates to a different feature, which has subsequently been disabled.
This is the internet (a series of tubes) and if people wander down unfathomable paths clicking on all the buttons as they go along - then the only thing which I can think of to combat it is to protect it (using authority to add or change stuff) and where this is too strict, then it needs to be monitored and the junk deleted and restored to what it was before.
I think some further analysis on this would be interesting, and Michael would be the correct person to present it.
On a related topic, I really like the idea which Mark Finnern has presented in this thread => Basic information in forums
Deleting and limiting people at SCN is not something I like doing, but to protect the "Expert Areas" I think that such a "Bazaar Forum" would be a very nice compromise. My opinion on this is based not on a Sunday morning cruise through the forums, but on trying to keep them clean for some time now.
I would be interested to know what you think of the idea, as you seem to have put some thought and concern into the quality aspects as well.
High quality, well researched contributions are the life-blood of SDN to keep knowledgable people interested. But we cannot ignore the freshers, who have other requirements and needs.
Cheers and thanks for stirring the pot a bit 🙂
Julius
Thank you for doing this spot quality analysis. It's worthwhile work on an important subject.
Just the fact that you took the time and raised the profile of quality is worthwhile as a reminder to all of us.
Quality is one of the components and determinants of the value we create in our communities (SDN, BPX, Business Objects...) and deliver to members, and it's a metric we measure in various ways (percent open threads, member satisfaction...).
The relatively small SAP Community Network team can't keep up with the flow alone (since we average 6,500 new forum posts in each of the 365 days of the year), so it's up to all of us as members to clean-up after ourselves.
Maybe this is a good new year's resolution as we start 2009: look back at our history and do a little tidying-up.
Thank you for your attention to the quality issue.
Regards,
Mark Yolton