In my not so humble opinion generating a graph like this from a database shouldn't require a 70 line ruby script!
mongohq + GS API + mongoPoll.rb on my VPS + emailReport2.rb + cron = awesome twice daily Thunderbird support report
mongohq + GS API + mongoPoll.rb on my VPS + emailReport2.rb + cron = awesome twice daily Thunderbird support report with trending tags, topics, anti-virus, ISPs and mail providers on posterous and in my inbox. Code is open source. Pull requests welcome!
- posting to posterous via email or the API results in truncated posts (I guess that's what happens when you have 2500 line long posts :-)! )
features that i intend to add:
- trending Thunderbird add-ons
- trending nouns that aren't: ISPs, Mail Providers, anti-virus, tags or add-ons
- remove polling, use Web Hooks instead
And just for fun here are the flickr pics of the slides:
I. Better Living
II. Just Say No
III.We Can Do This
IV. Support Comes to You!
V. Fork this!
Manual Support Forum browsing is soooo 1990s and a waste of support folks' time aka my Get Satisfaction Thunderbird API ruby and Mongo DB side project finally has born fruit :-) ! Seriously if you are manually searching your support forums and don't have customized support querying and reporting so vital, can't miss stuff comes to your support contributors via email (or some other semi-automated method like RSS or whatever) then you are doing support wrong.
I wrote a Ruby script to read via Get Satisfaction pseudo-REST API all the Thunderbird Support Topics into a MongoDB and wrote a ruby script to generate a custom email support report so cool, relevant, urgent stuff comes to me instead of me having to surf the forums.
Not only that I wrote a brute force search engine that allows searching the fulltext of all 40000 Get Satisfaction topics and tags via regexes.
Even cooler: Anybody who uses Get Satisfaction for support can use my code!!!!!!! All you have to do is:
- change "mozilla_messaging" to your Get Satisfaction company name
- change "mozilla_thunderbird" to your product name
- configure a gmail account config file called email.conf with 3 lines:
- p for your password
Finally, here's the README which serves as the documentation
(since this is a side project, the README is shall we say lacking :-); I hope to somehow get this into the SUMO project so that all support contributors not just me can use it; to this end I am writing a spec for this, fingers crossed)
Roland's utilities for Get Satisfaction Metrics and other GS fun stuff
- hardcoded for now for the Thunderbird product i.e. getsatisfaction.com/mozilla_message/products/thunderbird
- easily hackable for any other product
- forgive the barebones documentation for this project :-) !
- HELP WANTED: would love for a real developer to fix this code and enhance it
- email rtanglao AT mozilla.com if you are interested
- requires a MongoDB running on local host with a database called "gs"
- and with collection in the gs database called "topics"
Update the Mongo - ALWAYS do this first before running anything else!
./mongoUpdateSlurpGS.rb 2011 6 16 2011 6 19 2>mongoslurp.16-19june2011.818pm.stderr.txt &
Daily email metrics (I run this twice a day, typically 9a.m. and 4p.m. Pacific)
Weekly Metrics to CSV files and stdout (run once a week to generate support metrics for the Thunderbird Project Call)
./weeklyMetrics.rb 2011 6 13 2011 6 19 2>13-19june2011.stderr.txt 1>13-19june2011.stdout.txt
Search the Thunderbird Get Satisfaction MongoDB for regexes in the fulltext (title+content+replies) & tags
e.g. for Thunderbird 5 here's a sample search ("-r" is a comma separated list of regexes to search the fulltext for and "-s" is a comma separated list of regexes to search tags for)
./bruteforceSearch.rb 2011 6 1 2011 6 30 2>19june2011.tb5b1.bfs.stderr.1020pm.txt 1>19june2011.tb5b1.bfs.stdout.1020pm.txt -r tb5, "tb 5","thunderbird5","thunderbird 5",beta -s "tb 5",tb5,thunderbird5,"thunderbird 5","beta feedback",50,tb5,"tb 5",beta
Improve my MongoDB query to get the number of Thunderbird support topics created or replied to within a given time period please
I have a mongodb database created by my mongoUpdateSlurpGS.rb Ruby script with Thunderbird Get Satisfaction support topics from July 20, 2009 until the present (i.e. roughly 40000 topics) and I would like to know for a given time period how many support topics were created or modified. To calculate this, I wrote a second ruby script called alltopicsCreatedOrUpdated.rb (and the code is embedded below). My question is: is there really a need for a ruby script here.? Could the nested for "each" loop in alltopicsCreatedOrUpdated.rb be replaced by a couple of mongodb queries using the mongodb command line interface?
In the course of customizing Mozilla's Kitsune knowledge base website for Thunderbird (from the great Mozilla SUMO dev team, thanks jsocol and team!) software, I needed to change locale strings to reference Thunderbird instead of Firefox. Locale strings consist of:
- msgid e.g. "Firefox Help"
- followed by msgstr which is the localized version e.g. "Firefox-Hilfe" (for the German locale)
For Thunderbird I changed all occurences of Thunderbird to Firefox where there was a Thunderbird equivalent (I didn't change strings that have no Thunderbird equivalent e.g. there is no mobile version of Thunderbird so I left all mobile related strings unchanged). In the above example "Firefox-Hilfe" becomes "Thunderbird-Hilfe"
To do this automatically for all languages that the knowledge base for Thunderbird supports (which is dozens), I wrote a ruby script to do this: changeMessages-po-to-tb.rb
Of course this fails for languages where nouns are declined like Polish which in various situations uses "Firefoksa" instead of "Firefox" for example.
But it works for most of the locales and therefore has saved me and the localizers considerable time.
Is there a better way to do this? Perhaps a built in search and replace utilty in Linux or Mac OS X?
Mozilla happenings: Firefox 4 + Spaces party on April 15th + Thunderbird team now part of Mozilla Corp
Come join us for our big Firefox 4 party + Spaces party (how should we re-design our space to make it friendly for Open web compatible folks? We need your help figuring this out!). April 15, 5:30p.m. at 163 West Hastings 2nd Floor, RSVP on meetup, please! Free snacks, free drinks, free swag!
And I and the the rest of Mozilla Messaging have been integrated back into Mozilla corporation which means that after 2 Thunderbird releases where we established our ability to deliver and assembled a fine team, we'll have better access to Firefox folks and resources (David's Future of Messaging Blog post explains it better).
The cult of real-time metrics (in addition to big data) in Silicon Valley-style (Silicon Valley is now a state of mind and no longer just a place) startups now. Graph crucial and key metrics all the time and all in real-time ! I jest but it is valuable! Heretofore real-time metrics graphing has just been too hard unless you have the talent and experience of firms like Etsy to put the necessary infrastructure (graphing software, metrics collecting software, etc) in place.
However that difficulty is an opportunity that will be rapidly addressed (because real-time graphing is an itch that can be rapidly scratched by developers and it's helpful to companies' bottom line) and you can start to see this being addressed in projects like Simulchart.
In 1 hour I was able to get SimulChart graphing every 10 minutes the number of new Get Satisfaction Thunderbird topics using some node.js code (this is not a business critical real-time metric; i have plans for other real-time Thunderbird metrics, stay tuned :-) !). That's the future! 1 hour to get a business critical metric graphed in real time without having to stand up home grown infrastructure like Etsy and flickr did.
If anybody knows of any other real time open source and/or free graphing toolkits or services, let me know!
MongoDB is my current persistent data structure store of choice aka "how to slurp your Get Satisfaction data into MongoDB"
Got some data that you want to persist and manipulate but you are not sure what to do with it i.e. you are in exploratory or "side project" mode? Then MongoDB is a fine choice in 2011. Easy to get the data in and out and then when you have figured out what you want to do when you grow up :-) you can store it in a "real" database as well e.g. PostgreSQL. e.g. If you want to get your data out of Get Satisfaction or any other system with an API for backup and other purposes, check out my "hacked in an hour or two but I think it works", mongoUpdateSlurpGS ruby script. I used it to slurp in 20000 Get Satisfaction Thunderbird Topics and their associated replies and tags. If you want to do the same for your Get Satisfaction community, it's trivial to do: all you need is about 1 hour of a ruby (or other language,the code is pretty elementary) developer's time and some time to run the script.
It's so 1.0 :-) i.e. it's early days but it delivers on the promise of showing what's happening with the Get Satisfaction forum. Check it on gitbhub or if you running Firefox 3.6 or above click on the following link to run it from my dropbox (yay for dropbox deployment!):
Current Features as I write this blog post:
- Today's hot keywords (a bit mislabelled: should be today's hot email providers)
- Today's support pattern, a bar graph showing when folks have filed support topics and replies over the last calendar day
- 5 most active topics in the last 24 hours
- 5 latest created support topics
- 5 random topics from today
- 5 most recently solved topics
- 5 latest active solved topics (since we can't really close a topic, people can still comment on a solved topic so it's good to know this; this is a major Get Satisfaction gap IMHO, we need a a *real* close feature
- Real-time tag cloud - neat but not really helpful since most of the tags users use are not helpful
- applause feature when a support topic is marked close i.e. it plays an applause sound!!!!!
- Real-time - updated every two minutes!
Current bugs / feature requests
- improve the styling
- Add avatars of the folks who solved support topics
- The Get Satisfaction API could be improved with better documentation as well as support for real-time stats. Would be great for example to have an API call for the current calendar's day's solved topics, current contributors who solved topics etc instead of having to compute this (this is thinking out loud, haven't thought of this too much will blog about it more in the future)
- Deploy to github static pages instead of dropbox!