Sunday, February 27, 2011

How I created my weekly feed digest

So, if you read the frizzlefry blog, then you probably noticed I had an issue today that was all about my weekly shared Items in Google Reader.  I have been thinking about reorganizing all of my shared feeds into a series of digests.  The nice part, is that other people did all the work for me, so all I need to do is put their work together into a script I can run.

So, where to start?  ok, first I want to parse out an Atom feed, so I chose the Universal Feed Parser which is a nice little utility to parse out RSS/Atom feeds among other stuff I'm not currently interested in.  So, because the library I wanted to use is in Python, I decided that I was going to write a little python script to get the work done... I guess the language of a library is as good a reason as any to choose your language.  '

Anyway, ok, so how does this thing work?  really simple as it turns out...
To start off, I import the feedparser library, and create a feed object:

import feedparser

d = feedparser.parse('http://www.google.com/reader/public/atom/user%2F11424318483849164397%2Fstate%2Fcom.google%2Fbroadcast')


So, that creates a variable d, which is a feed object.  All I want from the feed is the title and description, so I can access those simply by using:


import feedparser

d = feedparser.parse('http://www.google.com/reader/public/atom/user%2F11424318483849164397%2Fstate%2Fcom.google%2Fbroadcast')

d.entries[itemnumber].title
d.entries[itemnumber].description


However, I want to print out those values, which can be done with the print like this:


import feedparser

d = feedparser.parse('http://www.google.com/reader/public/atom/user%2F11424318483849164397%2Fstate%2Fcom.google%2Fbroadcast')

print d.entries[itemnumber].title
print d.entries[itemnumber].description


Next, I want itemnumber to mean something, so I'll put the whole thing together in a for loop that prints all entries like this:


import feedparser

d = feedparser.parse('http://www.google.com/reader/public/atom/user%2F11424318483849164397%2Fstate%2Fcom.google%2Fbroadcast')

for itemnumber in range(0, len(d.entries)):
   print d.entries[itemnumber].title
   print d.entries[itemnumber].description


Now, I wanted to make sure that I limit the list to only include items after a given start date that I could specify as a command line argument, so I added a couple more lines:


import feedparser

startdate = datetime.strptime(sys.argv[1],'%Y-%m-%d').date()
d = feedparser.parse('http://www.google.com/reader/public/atom/user%2F11424318483849164397%2Fstate%2Fcom.google%2Fbroadcast')

for itemnumber in range(0, len(d.entries)):
   print d.entries[itemnumber].title
   print d.entries[itemnumber].description
   if datetime.strptime(d.entries[itemnumber+1].published,'%Y-%m-%dT%H:%M:%SZ').date() < startdate: break


So, I created the startdate variable, which is a date provided by the command line in the format
--, then it breaks the loop if date published is before (less than) startdate. 

Finally, I wanted to add some basic HTML formatting to make it easy for me to copy and paste the text into blogger... that lead me to my final version of the python script, which looks a little like this:


import feedparser
from datetime import date
from datetime import datetime
import sys

print sys.argv[1]
startdate = datetime.strptime(sys.argv[1],'%Y-%m-%d').date()

d = feedparser.parse('http://www.google.com/reader/public/atom/user%2F11424318483849164397%2Fstate%2Fcom.google%2Fbroadcast')
print '<html><head><title>'

print d.feed.title
print '</title></head><body>'

for itemnumber in range(0,len(d.entries)):

print '<h1>'
print d.entries[itemnumber].title
print '</h1>'
print d.entries[itemnumber].description
print '<br /><br />Date Shared: '
print datetime.strptime(d.entries[itemnumber].published,'%Y-%m-%dT%H:%M:%SZ').date()

if datetime.strptime(d.entries[itemnumber+1].published,'%Y-%m-%dT%H:%M:%SZ').date() < startdate: break
print '<hr />'
print '<hr />'
print '<hr />'

print '</body></html>'


So, the first part of my script was done... I named the file GeneratePage.py, and proceeded to create three more files... the second file was SaveDate.py, which would simply output the date in the format I needed it using two lines:

from datetime import date
print date.today()


I figured the best way to track the previous time the script was run would be to save the date to a file, so I created a file called date.txt that contained only the string "2011-02-26".

Then I created a batch script that would tie everything together... the batch script looks like this:

#!/bin/bash
D=`cat date.txt`
python GeneratePage.py $D
python SaveDate.py>date.txt


This script would set the date from date.txt to a variable called D. Then it would run the GeneratePage.py script while passing it the value of D. Then it would run the SaveDate.py script, with the output replacing the contents of date.txt.

At this point, the script could be run every week, and the output can be copied and pasted into blogger.

The next step will be to automate the process of putting the dialog onto the site, which will actually take a little effort, but I'm pretty sure someone else has already done the work for me, I just have to figure out who did it.

Oh, on one other note, in case you are curious why I didn't just redirect all the output to a file, it's because some of the feed items use Unicode characters that cannot be converted to ASCII... so Bash gets angry when you do that.  I might be able to fix this easily, however, I really don't mind having to copy and paste the characters into the blog anyway.

Later,

     SteveO

Thursday, May 6, 2010

Managing your Droid Incredible with Ubuntu 10.04 Linux

Hey everybody!  It sure has been a while since I updated this blog, but I think it's time I did.  Last week I picked up an HTC Droid Incredible, which is the Verizon version of the Incredible, and I've found that nobody has really written about what you can do with this thing using Ubuntu (a Debian-based Linux distribution.)  So, I'm here to tell you that you don't need a Windows PC to manage your device, and when I say that, I'm not implying you have to be a Linux expert or manually put all your files on the device.  So, to avoid most of that cryptic Linux stuff, I'm going to almost completely avoid the command line (I'll tell you the command, but I'll also tell you how to do it without the command...except for DVD playback, you have to copy and paste that command.)  I will also assume you have a default Ubuntu installation, so you may think I'm just being silly by installing some of this stuff.

Please note that I won't cover the mobile broadband functionality because the setup I have for it is very complicated and unstable at the moment.  Once I figure out a simple way for people to get it working, then I will write a post about it.

The super duper quick version



For those of you who know Ubuntu/Linux well, just install and configure the Restricted Extras Packages, libdvdcss2, Banshee, and Arista with this command (All of it is in the Ubuntu Software Repositories):

sudo apt-get install ubuntu-restricted-extras kubuntu-restricted-extras xubuntu-restricted-extras libdvdcss2 banshee arista; sudo /usr/share/doc/libdvdread4/./install-css.sh

Then you're setup is done.  Go to the section about Banshee if you want your device to display with the correct name and use the correct folder, and Arista is so simple, all you have to do is: choose a video file or DVD device as the source, select Ipod as your device, and Low as the Preset, then click Add to Queue (I can't really tell the difference between the low and high presets when using the device, but low uses half the memory.)


Codecs and DVD Playback

     So, let's spend a little time getting your system ready, and then our entire experience will be a little nicer.  First things first, you will want codecs for videos and music.  So, open up the Ubuntu Software Center (Applications -> Ubuntu Software Center), then type "restricted" into the search bar in the upper right corner.  You will see three packages that you want*: Ubuntu Restricted Extras, Kubuntu Restricted Extras, and Xubuntu Restricted Extras.  I don't think you really need all of them, but I always just install all three, so I'm going to tell you to do the same.

* Please note that the Restricted Extras Packages and the libdvdcss2 package are not allowed in some countries.  My primary audience is people in the US.

Ok, I kinda lied a little about the cryptic stuff thing... If you want to play back DVDs, you need to do one thing:

  1. Press Alt+F2 on the keyboard, and the Run Application dialog pops up.
  2. copy and paste this command into the Run Application dialog:
    "sudo apt-get install libdvdcss2; sudo /usr/share/doc/libdvdread4/./install-css.sh"
  3. Check the Run in terminal checkbox
  4. Click Run.
  5. A window with a bunch of text pops up, and I think you might have to hit "y" to accept the license agreement.
So, that wasn't too bad was it?  Because of restricted licenses, Canonical can't just distribute the DVD playback libraries with Ubuntu, so you have to get that separately and accept the EULA (End User License Agreement.)  Then you have to run a script that sets everything up for you; that is what the above command does.

Ok, now everything is set up, and you are ready to play all your music, watch movies, and all that kind of stuff.  

Media Management Software

Now, there are two more programs that you will want:  Banshee Media Player, and Arista Transcoder.  Both of these are available in the Ubuntu Software Center, so just type them both into the search box (one at a time of coarse) and you have all the software you need.  There is just one thing though, Banshee detects your Android Phone as a generic Android Device.  I don't mind the picture of the G1, but I just want the program to use the default music folder (/emmc/MP3/ for internal storage, or /sdcard/MP3/ for the microSD card), and I want the correct name for my phone.  This is actually a really simple thing to do.   You just create a text file, and save it to the device with a very specific name.

Get Ubuntu to See your Phone

By default, my phone goes into charge only mode when I plug it into a USB port, which you can change, but I like it that way.  So first, connect your phone to your computer.  Now, on the phone, open the notification area, tap the USB Icon that says "Charging Only" next to it, then select Disk Drive from the menu that appears.  Once your phone appears on your computer, you need to save a file into the root folder of the device.

The root folder would typically be called something like C:, D:, E:, etc... on a Windows computer.  You can recognize it because it shows up as a USB stick on the Places menu and on the side bar in the file browser (or in this case the save as window.)

Also note that if you have a MicroSD card, that will show up as a separate drive/device, You can set the name for the phone and MicroSD card separately in Banshee by repeating the following steps with a different name for the memory card.  You have to manage both of them separately.  Most software in the Android marketplace will not recognize media on the phone's internal storage yet, but if you just use MP3s and Ipod-formatted videos, you can just use the default media software on the phone.

Get Banshee to show the correct Name for your device:


Now that your device is connected and Ubunti can see it, you can save a text file to the device.  So, Open a text editor (Applications -> Accessories -> gedit Text Editor.) Then type two Lines (just copy the stuff between the quotation marks):
  • Line 1: "name=Droid Incredible"
        - This name can be what ever you want Banshee to call the device
  • Line 2: "audio_folders=MP3/"
        - This is an optional line; I chose to add it because I had already manually copied files to the MP3 folder, and now I wanted to keep everything simple.
Ok, now save that file as (File->Save As...) ".is_audio_player" and put it on the root of your device or SD card.  (You need the period at the beginning of the file, and the file has no extension at the end)  

Note that from now on, this file will be hidden, so to view the file in the file browser, you need to select View -> Show Hidden Files (or Press Ctrl+H.)

Ok, so now Banshee should show the name you specified, any music you have already put into the default folder, and sync new music to the folder you specified.

If you want a complete list of what you can put into the .is_audio_player file, you can refer to this page:

The Floccinaucinihilipilification Homepage - .is_audio_player file format by Darin Ohashi


Formatting Video for Your Device

So, after spending hours of trying different video transcoders, and different video players, I found that Arista Transcoder is the easiest option available.  You just open the program, choose a source, choose a device, choose a preset, and add to queue.  You can even queue up several video files and let them all go.  Then, when all is said and done, you can put the media on your SD card or the internal storage and play it through the photo browsing tool that came pre-loaded on the phone.

The source is either a DVD drive or video file.  For the device, just choose Ipod.  For the preset, choose low.  You can use High, but you will need to stop all non-essential processes or the video can go out of sync if you get an email or something.  Plus, the High preset uses twice as much memory, and I think about 200MB is much more reasonable than about 500MB.

Using Banshee

Banshee is really easy to use.  It will download album art, but it doesn't use the correct format for the HTC Incredible.  If you really need your album art, I have been using Media Monkey on Windows to update Album art directly on the device, and that works pretty well... I'll have a Linux solution soon, but that's not really a priority for me.

I've also found that playlists don't quite work right "out of the box."  I'll figure that out later, because I normally just press shuffle all, then stop looking at the device until I want to skip a song.

As far as video is concerned, you can just import everything to the library from the file menu.

At this point, you can use Banshee to sync all of your music and videos to your device in a very similar way to how you would sync everything with your iPod or other devices.  In fact, you can even use Banshee to sync to your iPod!  So, I hope this helps make every Ubuntu user that much happier with Android.

Check back soon to see updates on my favorite Android Apps, Linux tips, and general Tech stuff.

Later,

     SteveO

Thursday, July 23, 2009

Google Wave First Impressions

So, I got an invite to the Google Wave Developer preview, and it's pretty cool.  It looks right on track for a beta at the end of September. It was a little funny at first because you just get this information overload right away.  Everything everyone was talking about flooded my inbox.  This was because of a special dev channel that you can subscribe to right when you sign up for your developer account.  Basically it's a constant stream of what the devs think is important to get done.  There is a lot going on.  Linking Wave to Google's App engine works pretty nice, and you can get a lot done with robots. So, dealing with the information overload lead me to use the search box a lot.  There was no way for me to read all the waves, they all just kept going.  Each wave would have a dozen or so people doing all sorts of things.  One wave was a group of people collaboratively creating a book, and another wave was a list of robots.  The most common waves I found were titled "test"  because that was what the channel was for.  Testing stuff that you are writing for Wave. Once you get used to the information just streaming in from everywhere, you start to like it.  You can search for anything you want, and real time results of waves on your channels start flowing in.  You can reply to waves, play games of chess, and even add hot spots to a map.  The conversation really just flows smoothly. Now that I'm getting in my groove, I started to wonder what kinds of cool stuff I can do with Wave.  I started looking for bots, and the robot wave just popped right up.  I saw the Twitter bot, Tweety the Twitbot, and then added the email address (tweety-wave@appspot.com) to my contact list.  The whole process to this point was just like any other bot (like smarterchild on AIM...)  Now I can start a wave, and include tweety.  From then on, anything you say in the Wave will end up on twitter.  So it's important not to use multiple bots in one wave... like I just did... you end up tweeting things like $MSFT... anyway, that tweet was supposed to be a wave for Stocky (stocky-wave@appspot.com) who actually changes the text you enter into a stock quote (like you type "$MSFT" and Stocky changes it to this "MSFT (25.56)".) The bots are awesome!  You can embed pictures, maps, surveys, polls, real time weather, HTML, and all sorts of stuff into a Wave.
Another cool thing I noticed right away is that everything can be minimized.  With the announcement of the Google Chrome OS, I can see Wave as evolving into some sort of live desktop.  You can have multiple waves open at once, and just switch between them.  The full screen reading mode will also be pretty nice for my netbook. In final, my first impression is that Google Wave will revolutionize the way people communicate on the web.  Everything is so easy to do.  Want the weather?  tell the weather robot where you want the weather for.  Stocks? Directions?  Microblogging?  Actually that makes me think, how will this effect twitter?  You have real time search, and a more interactive way to communicate with your friends! Anyway.  I was really excited about Wave before, and now that I've gotten a taste of it, I don't think I can stop.

Wednesday, June 17, 2009

Social Media is definately the future!



It should not surprise anyone that I've been inspired by another TED talk.  I mean, every talk on the site is something worth watching at least once.  This one is about social media.  I have been following Andrew Sullivan on his incredible coverage of the protests in Iran, and I have been watching the #iranelection tag on twitter too, and I'm a firm believer that CNN doesn't stand a chance against this type of media.  Sure, the video isn't in HD, and there isn't a reporter editing the footage and choosing which scenes to show you... it's just what is happening.  The real coverage.  It worked in China for the earthquake... it was used to make sure that the schools to replace those that collapsed were actually built to code... and it's the only media coming out of Iran while all the news stations are being cut out of the loop.

     I know you aren't going to get the best quality journalism from this news, but you aren't going to get good quality journalism until after the issue is resolved... the only thing that TV networks can ever offer you is coverage, and any monkey with a camera can do that.  I don't mean to say camera men aren't talented because finding the right shot isn't easy, but most of the difference between a camera man and some ordinary person is that the camera man usually has better equipment.

     This leads me to my next realization, the future of print media.  So, we all know the newspapers are having trouble re-acclimating to the digital world.  When newspapers move to the internet, every article has to carry it's own weight... the atomic unit of sales changed from an entire paper, to an individual articles.  Social media is really good a producing raw unedited footage of what is happening, and initial public reactions, however traditional media has this ability to censer the story, check the sources, and produce good quality media that requires investment to produce.  We must not loose that.

   My question, how do we combine our social media and traditional media into a unit that works together?  I like the concept of free licenses for social media (free as in freedom to reuse or remix the content for non-commercial or at least non-profit use), but the real question is how do you put a price on or handle purchasing licenses to the media?  I feel the creator has more right to profit from their work than some media outlet using the story to attract viewers or sell papers.

Anyway, there is a lot that can be said for the way the world is shifting to the internet.  Blogs, news aggregation, youtube, cellphone cameras, and personal video cameras are changing the way we get out news.  Which I think is fine.  I think that spreading the information is very important, but there is something to be said for the people who come in after the story is over.

The people who follow up the story with quality journalism (the who, why, what happened, and how it turned out.)  The story that is documented for the historians to record, and the story our descendants will learn in school.  We need to make sure that someone checked the score board and made sure they weren't mislead by someone with an agenda.

So, embrace the digital revolution, but don't forget your roots.  Support quality journalism (like PBS.)

A call for opinion!

So, I want to know what everyone thinks of the Wikidot site I'm working on.  I want to know what you want to see on the site.  Right now, it's my super-spam RSS feed, which has all of my activity from Hulu, Netflix, Wakoopa, Twitter, and my blogs all in one place.

Friday, June 12, 2009

The Smart phone competition!

So, over on Gizmodo.com they did a review of the latest and greatest smart phones, and not much to my surprise, Windows Mobile wasn't on the list!  haha.  Ok, I am biased against Windows, which is the only reason I say that; in all reality, Windows mobile is a decent mobile operating system.

iPhone 3.0 VS Android 1.5

Anyway, the review put my favorite phone OS (Android) against my arch nemesis (iPhone).  So I have to bask in this little bit of glory.  Both phone operating systems are nearly the same in terms of highlighted features in the review, with the exception that Android doesn't work with iTunes, Android doesn't officially support tethering (even though the iPhone carrier doesn't support it), and Android supports both background apps and third-party  Apps.  I do have to say that I am disappointed that the review didn't mention that Android supports the ability for user's to add third party video codecs, which iPhone doesn't.

iPhone 3G S VS. HTC Magic

On a hardware level, the iPhone has multi-touch and a headphone port, and I have to admit I would really like to see both of those things added to an Android phone.  The plus side is that anyone can make an Android phone, so one day it is possible to add multi-touch to a phone for Android.  On the Plus side, the magic has a microSD card slot, which gives it the potential for more than 32GB of storage, and the Magic has a 3.2 mega pixel camera, which is slightly more than the iPhone.

Cost

So, both phones are "linked" to a subscription.  The difference in cost between the two phones is fairly significant right off the bat.  With a 2 year contract, the iPhone is either $200 or $300, while the HTC Magic is $150.  The iPhone is permanently linked to AT&T, however the Magic is only linked to T-mobile for the contract, and after your contract, you can take an Android phone to another phone provider that uses SIM cards.  Then in terms of your cost over the next two years, the iPhone's average cost is $290-$390 more than Android, and the maximum cost is $750 - $850 more for the iPhone.  In fact, between the iPhone 3G S, HTC Magic, Blackberry Storm, and the Palm Pre, the average cost for the Magic is less than any other smart phone, and only the Palm Pre offers a cheaper plan if you compare the unlimited everything plans.

So, which phone do you think I'm going to get?  To me it's pretty obvious because I decided I would get an Android phone last year, and I will honestly probably hold off until next year to get an Android phone just because I want to see if Verizon is ever going to jump on the bandwagon.  I honestly have had some bad luck with T-Mobile over the past couple years, and I am holding out for multi-touch and an 1/8" jack.

Anyway, I am biased toward Android, and the iPhone is a really good phone;  That's why they are my arch nemesis.

Tuesday, June 9, 2009

Smart power outlets? Awesome!



So, I have to admit that this video does kinda seem like it's just promoting a product, but it's a totally awesome idea!

If you aren't there to use power, you can turn your outlets off!

Anyway... just some food for thought.