You are here

ruby

Best MongoDB to CSV tool to generate CSV files for graphing using a JS toolkit, Excel, etc?

Submitted by Roland on Thu, 2012-08-09 23:11

My previous post was unclear. I'll try again. It shouldn't take a 70 line Ruby script or indeed procedural programming to generate a CSV file from a MongoDB database. So Lazy web :-) what's the best Best MongoDB to CSV tool to generate CSV files  for graphing using one of the JS toolkits, Excel, etc so I don't have to write a 70 line Ruby script like getOneDayAnswerRate.rb?

Thunderbird - Get Satisfaction Answer Rate - A tale of tools; this shouldn't require programming

Submitted by Roland on Thu, 2012-07-26 18:06

mongohq + GS API + mongoPoll.rb on my VPS + emailReport2.rb + cron = awesome twice daily Thunderbird support report

Submitted by Roland on Fri, 2011-12-23 16:15

mongohq  + GS APImongoPoll.rb on my VPS + emailReport2.rb + cron = awesome twice daily Thunderbird support report with trending tags, topics, anti-virus, ISPs and mail providers on posterous and in my inbox. Code is open source. Pull requests welcome!

Bugs

  1. posting to posterous via email or the API results in truncated posts (I guess that's what happens when you have 2500 line long posts :-)! )

features that i intend to add:

  1. trending Thunderbird add-ons
  2. trending nouns that aren't: ISPs, Mail Providers, anti-virus, tags or add-ons
  3. remove polling, use Web Hooks instead

"sudo gem install SystemTimer" fixed my mongodb timeout issues after time machine restore

Submitted by Roland on Sat, 2011-12-17 23:42

don't know why time machine restore didn't fully restore my Mac OS X Lion ruby install but who cares :-) ! And don't know how I figured this out. Other than looking at  the yakshaving to get ruby working with mongodb on my VPS.

Better Support Living through Software Presentation

Submitted by Roland on Wed, 2011-06-29 23:30

Presentation created for June 30, 2011 Vancouver Ruby / Rails meetup (flickr set, slideshare, pdf)

See my Better Support Living through Software for the full details

And just for fun here are the flickr pics of the slides:

I. Better Living

BETTER SUPPORT LIVING THROUGH SOFTWARE

II. Just Say No

Just say no aimless support forum browsing

III.We Can Do This

We Can Do This with: Ruby, Get Satisfaction API, Mongodb

IV. Support Comes to You!

Support Comes To You !

V. Fork this!

Fork This!

AttachmentSize
BetterSupportLivingThroughSoftware.pdf348.82 KB

Improve my MongoDB query to get the number of Thunderbird support topics created or replied to within a given time period please

Submitted by Roland on Wed, 2011-05-11 14:27

I have a mongodb  database created by my mongoUpdateSlurpGS.rb Ruby script with Thunderbird Get Satisfaction support topics from July 20, 2009 until the present (i.e. roughly 40000 topics) and I would like to know for a given time period how many support topics were created or modified. To calculate this, I wrote a second ruby script called alltopicsCreatedOrUpdated.rb (and the code is embedded below). My question is: is there really a need for a ruby script here.? Could the nested for "each" loop in alltopicsCreatedOrUpdated.rb  be replaced by a couple of mongodb queries using the mongodb command line interface?

How to find Get Satisfaction topics by tag using the Get Satisfaction JSON REST-like API

Submitted by Roland on Fri, 2011-05-06 07:30

Since the Get Satisfaction developer docs are shall we say in need of some love, here's an ruby irb session (exercise for the reader, convert to a python or lisp REPL session :-) !) that illustrates how to find all Get Satisfaction topics tagged Android (to play along and find the get satisfaction topics tagged "android" for your company, make sure you have ruby installed on Linux or Mac OS X  along with the json ruby gem and substitute your company name for "mozilla_messaging"):

Create your own custom daily Get Satisfaction email report with emailReport.rb

Submitted by Roland on Mon, 2010-08-02 23:42

UPDATE November 2011:

The code has been dramatically re-factored so i deleted it from the gist. Please use the revised code at:
https://github.com/rtanglao/momogs/blob/master/emailReport-m.rb

For my technical support lead job at Mozilla Messaging, I continue to try to find the best way to sort and sift and help out folks who need support and identify the hot issues. We currently use Get Satisfaction for Thunderbird support and I subscribe via email to everything but it's really difficult to read through the over 200 emails a day that Get Satisfaction sends

So I developed over the last 9 months a series of Ruby scripts to see the most active support topics (getactive.rb),  the contributors who are actually being marked by the community of users as solving a support topic (getcontributors.rb), who's doing the most replies (gethappykludged.rb) and pick out some random support topics (getrandom.rb)

Up until last week I have been using these scripts on a manual basis but not as consistently as I would like since it's pain to run all four manually.

To make a single email report that ties together everything in one nice summary, last week I finally took 3 hours and wrote emailReport.rb (code after the jump). It's crude but it works and is a useful report that I run every day now for the previous day (e.g. on Monday morning, I run it for Sunday and Saturday). It runs all the above mentioned scripts and then emails a summary email:

Here's an example of how you invoke it:

(for August 2, 2010)

./emailReport.rb 2010 8 2 2010 8 2 2>emailreport.20100802.20100802.stderr 1>emailreport.2010082002.20100802.stdout

And here's a sample email:

From: xxxx@gmail.com
To: yyyy@zzzzzo.com
subject: MoMo Support Report FROM: 2010.8.2 TO: 2010.8.2
Date: Mon, 02 Aug 2010 23:31:16 -0700
Get Satisfaction Top 5 active:
5,http://getsatisfaction.com/mozilla_messaging/topics/how_do_i_export_folders_as_mbox_files_thanks_chris
5,http://getsatisfaction.com/mozilla_messaging/topics/migration_of_thunderbird_data_from_old_to_new_computer
4,http://getsatisfaction.com/mozilla_messaging/topics/delivering_mail_98_thunderbird_problem
4,http://getsatisfaction.com/mozilla_messaging/topics/from_thunderbird_to_outlook_how
4,http://getsatisfaction.com/mozilla_messaging/topics/cannot_start_thunderbird_already_runnning

Get Satisfaction Contributors:


Top 10 Get Satisfaction Repliers:
2, cam_1319937
3, simonpaquet
3, ankitgupta1980
3, davro
3, pat_drummond
3, kent_james
4, bhuvnesh_thakar
5, wayne_mery
9, archaeopteryx
30, michael_a_pasek


5 Random Get Satisfaction Topics:
http://getsatisfaction.com/mozilla_messaging/topics/crashing_after_start_up
http://getsatisfaction.com/mozilla_messaging/topics/problems_editing_an_email_to_be_forwarded
http://getsatisfaction.com/mozilla_messaging/topics/dmg_file_whats_this
http://getsatisfaction.com/mozilla_messaging/topics/want_to_use_thunderbird_for_mass_mailing
http://getsatisfaction.com/mozilla_messaging/topics/pasting_images_takes_minutes

What I did well

  • It works and when it doesn't I can check stderr for diagnostics and usually quickly figure out what went wrong (Get Satisfaction has a known bug on topics that have URLS that begin with numbers and also JSON can't cope when folks paste binary data into support topics; fortunately both cases are rare -:-) !)

What could be improved

  • The JSON API is called on the same data four times, once for each script (i.e. I don't store any of the GS data) so it takes 5-10 minutes to run emailReport.rb!
  • It would be more efficient to just read the data into a database, e.g. couchdb and then have the four scripts read from the database
  • Ruby is not an excellent choice for Windows users; would be better to write this in JavaScript and then anybody (i.e. other support contributors not just folks on Mac OS X or Linux) could use it
  • Badly need to way to extract the GS conversations I am involved in and highlight the top 5
  • Although a separate can of worms :-), I would love to have a really great search engine that works on the GS support data; unfortunately the GS management view search doesn't work so well.

Help Wanted

I'd love some help :-) (what's in it for you! Open Source Whuffie, fame and adventure :-) !!!)   in the following areas:

  • Convert to JavaScript and to a web app so all can use it :-) !
  • Put the GS support data into a JSON friendly database like CouchDB

 

The Code

The code has been dramatically re-factored so i deleted it from the gist. Please use the revised code at:
https://github.com/rtanglao/momogs/blob/master/emailReport-m.rb

 

My ideal mobile mad scientist language

Submitted by Roland on Tue, 2009-11-24 22:35

After some digging and research around the web, my ideal mobile mad scientist programming language would:

  • have the 2D and 3D graphic manipulation power of Processing, Nodebox and Shoes
  • be cross platform mac, windows, linux, maemo on mobile, iPhone, android
  • be 'web native' i.e. REST, JSON, XML and all the other web API stuff built in and not bolted on like it is Processing, trying to use the flickr api from Processing is shall we say kludge-o-rama (awesome code from bryan chung but indicative of the unnecessary struggle one is forced to engage with in Processing and other non web native languages)
  • not use a Java-like syntax, death to curly braces and wasted semi-colons
  • be dynamic, death to the Java/C++ cargo cult of typing for no reason 
  • be easily adaptable to new APIs and new sensors through the ability to create a domain specific language and/or easy to use and beautiful foreign function interface
  • be open source, sorry but for my mobile art,  i can't use programming environments and languages that are not open source
  • support the REAL loop, I don't want to spawn threads for the sake of questionable 'concurrency' like I am forced to with OSGI and the Bug Labs Bug

IF I were an idealist that pretty much rules out everything :-)

Fortunately I am a pragamatist. So I will continue my experiments in:

  • Nodebox & Python on the Mac
  • Cocoa Touch and Objective C on the iPhone

What about Processing? Sorry can't handle the Java syntax and the pain of doing XML and JSON and REST programming and the kludge-o-matic way to access Java libraries. processing.js? too early and too much impedance mismatch to use all the lovely JS libraries out there. And Shoes is promising especially if it were improved so you could easily use normal Ruby gems but given its current "hibernation" "post-Why" not sure it will continue to be improved.

What should I use on Maemo if/when I get an N900? Ruby plus SWIG or some such foreign function kludge er interface :-) to access the sensor APIs which I assume are only available in C and C++ ?

What should I use on Android if/when I get an Android device?

What should I use on Windows? Not that I really care :-) But it would be lovely to have Windows people join in my fun without having to do anyting. Eines Tages!

Somehow I think the "mainstream" world is moving towards my ideal solution and the mainstream solution for what I want will look more like processing.js and ruby-processing or smalltalk i.e. scratch then it will look like Processing, Nodebox or CocoaTouch

REAL - Read, Evaluate, Art, Loop - N900 is the closest thing to a "REAL" machine

Submitted by Roland on Thu, 2009-10-15 23:19

What I am looking for in my ongoing mobile art experiments is REAL!

  1. Read the sensors (GPS, accelerometer, compass, etc)
  2. Evaluate the data from the sensors
  3. Art - Make some art (sound, graphic, image, etc) and display and store it on the mobile
  4. Loop - Back to step 1

And I want to do it in a dynamic environment that doesn't force me to do yak shaving like spawning separate threads for each of the sensors or other such needless complexity that's not needed by my artistic algorithms. Nor the slings and arrows of outrageous certificates or certifications or developer programs or DRM malarkey :-) !

After reading the N900 technical reviews from the Maemo summit, it appears that the unlocked version of the N900 is the closest current device that could do this:

  1. It's Linux so assuming the sensor APIs are available from a standard Linux C (i.e. not using some non standard craziness like Carbide C++ for Symbian) library, Python, Ruby, name your favourite dynamic language here, etc (C++ and Java are just not malleable and easily hackable enough, sorry!) bindings could be (and probably are or are in the process of being) built to those C libraries
  2. No need for developer fees or ridiculous certificates
  3. It has all the sensors I want for my current experiments
  4. It's NOT mass market, but it's mass market enough for hackers (unlike Bug Labs Bug which I love but already probably has less than 1/10 the amount of developers working on it as the N900).

Am I right? Time for me to watch the blogs for signs of the N900 and Maemo 5 and 6 making dynamic languages first class citizens unlike on Symbian where S60 Python was far too many steps behind Carbide C++  (and time for to save up for the N910 since the N900 will probably be crippled in some significant way as all 1st gen Nokia devices are e.g. N95-1 not having enough memory)

My first ruby program - a program to animate my 1796 Skytrain Photos from flickr

Submitted by Roland on Wed, 2009-01-14 20:01

Let's say you had 1796 photos in a directory from flickr and wanted to animate them in Shoes which is built on Ruby, how would you do it?

Here's how I did it (forgive my bad code, it was just a fun 1st project and of course you could do this in QuickTime Pro with no programming but that wouldn't be as much "fun" :-) !):

Subscribe to RSS - ruby