Busy making things: github, links, photos, @mc.

Geolocation Information Gaps

Posted: August 5th, 2005 | Author: Matt Croydon | Filed under: Projects, Python | 96 Comments »

I’ve been seeking out interesting data sources to plot in Google Earth after learning the basics of KML. I’ve been wanting to do something cool with NOAA’s XML weather feeds since I heard about them, so I thought I would download the 700kb list of stations serving up XML and spit out some KML from that data as a “neat” first step.

I’ll probably still do that, but after parsing the data, I’m a bit dissapointed. As always there are huge gaps in geolocation information. In order to get my hands on the data I turned to xmltramp which is an awesome library for accessing simple XML documents in a pythonic way. I then whipped up a few lines of Python to walk through the data:

import xmltramp # http://www.aaronsw.com/2002/xmltramp/

f=open('stations.xml', 'r')
doc=xmltramp.parse(f.read())
count = 0
total = 0
for station in doc['station':]:
  total = total + 1
  sid = str(station['station_id'])
  lat = str(station['latitude'])
  lon = str(station['longitude'])
  if (lat != 'NA') and (lon != 'NA'):
    print "Station ID: " + sid + \
    " (" + lat + "," + lon + ")"
    count = count + 1
print str(count) + " out of " + str(total) + \
" stations are geolocated."

Here’s the output of the above code:

mcroydon@mobilematt:~/py/kmlist$ python kmlist.py
Station ID: PAGM (63.46N,171.44W)
[... snip ...]
Station ID: KSHR (44.46.10N,106.58.08W)
422 out of 1775 stations are geolocated.

Well that’s a bummer. 422 out of 1775, or less than 25% of all stations are geolocated. While that’s still 422 more stations than I knew about previously, it’s a far cry from a majority of weather stations across the United States.

Another thing you will notice is that some stations appear to be expressed in degrees in decimal form (63.46N) while others appear to use Degrees/Minutes/Seconds (44.46.10N).

It’s gaps like these that can make working with “found” geolocation data frustrating.

96 Comments on “Geolocation Information Gaps”

  1. 1 Sean Gillies said at 9:20 am on August 5th, 2005:

    Matt,

    Try http://weather.noaa.gov/tg/site.shtml. I’m not sure it has everything you need, but has a couple big text files of site locations.

  2. 2 Matt Croydon said at 9:41 am on August 5th, 2005:

    Thanks Sean. I think something like this http://www.rap.ucar.edu/weather/surface/stations.txt should allow me to fill in the gaps. wc -l counts 8509 anyway, so here’s hoping.

  3. 3 Sean Gillies said at 10:51 am on August 5th, 2005:

    Daryl Herzmann, a WX/GIS/MapServer guy at Iowa State, has a bunch of weather-related OGC web services: http://mesonet.agron.iastate.edu/ogc/. The CONUS NEXRAD WMS you should be able to hack directly into your Google maps.

  4. 4 Simon Chan said at 10:52 am on August 10th, 2005:

    The value 63.46N is not in decimal form; it is Degrees/Minutes and without Seconds reading.

  5. 5 Matt Croydon said at 11:22 am on August 10th, 2005:

    Thanks for the heads up Simon.

  6. 6 Term Insurance said at 6:39 pm on January 21st, 2007:

    Term Insurance…

    contumacy ventricle caters.Life Covarage [url=http://www.tucsonwildlife.org/#]Life Covarage[/url] http://www.tucsonwildlife.org/# …

  7. 7 Black Jack said at 5:30 pm on January 22nd, 2007:

    Black Jack…

    trudged.dirtiness canvassed maximums!cauldron.frieze Black Jack [url=http://www.anblackjack.com/#]Black Jack[/url] http://www.anblackjack.com/# …

  8. 8 Home Insurance said at 11:30 am on January 28th, 2007:

    Home Insurance…

    Callan how Zaire tanker Home Insurance [url=http://www.homeformule1.com/#]Home Insurance[/url] http://www.homeformule1.com/# …

  9. 9 91d75aead549 said at 6:19 am on May 15th, 2008:

    91d75aead549…

    91d75aead549c7b8d134…

  10. 10 Yoppi Do said at 11:48 am on January 28th, 2010:

    It’s gaps like these that can make working with
    http://westblog.ru/?p=140
    “found” geolocation data frustrating.

  11. 11 Anonymous said at 4:02 am on August 23rd, 2010:

    Switch Home Insurance…

    Home Insurance The easiest way to make sure it has never been in a situation where you try to switch insurance companies to discover the new company does not know what it is to ensure the houses most common reasons are insurable. Dog Breeds Home insura…

  12. 12 Alfredo Polter said at 2:46 pm on November 8th, 2010:

    oversize diary you’ve include

  13. 13 Melynda Capestro said at 7:07 pm on November 12th, 2010:

    It report details how to take care of.

  14. 14 Salvatore Nido said at 12:12 am on November 13th, 2010:

    You do not want to risk your credibility in order that this has been a riches to rags story.

  15. 15 Long Desrochers said at 7:15 am on November 16th, 2010:

    Like your website, I have been reading quite a lot about this recently and it is helpful to get lots of unique bits of info and points of view. Look forward to finding out more.

  16. 16 inflatable rafts said at 10:49 pm on November 25th, 2010:

    E. M. Forster~ We must be willing to let go of the life we have planned so as to have the life that is waiting for us.

  17. 17 stearns inflatable canoes said at 11:43 pm on November 25th, 2010:

    Brother: Did you put the cat out? Sister: Why, is it on fire?

  18. 18 tekne kiralama said at 9:58 pm on December 1st, 2010:

    very good \o/

  19. 19 Stephane Aubin said at 11:18 pm on December 27th, 2010:

    I really like your blog and i respect your work. I’ll be a frequent visitor. Some of my younger friends lean more towards such singers as George Strait and Garth Brooks.

  20. 20 weight loss pills said at 11:37 pm on January 19th, 2011:

    Unusual this publish is totaly unrelated to what I used to be looking google for, but it was once indexed on the first page. I guess your doing one thing proper if Google likes you enough to position you on the first web page of a non comparable search.

  21. 21 Small Bathroom Renovations said at 7:58 am on January 20th, 2011:

    Please, keep up the awesome work and continue to post topics like this. I am really fan of your site.

  22. 22 Andrew Warnix said at 2:14 am on January 21st, 2011:

    I dugg some of you post as I cogitated they were very helpful invaluable

  23. 23 Healthmaster said at 11:14 am on January 21st, 2011:

    Extraordinary this publish is totaly unrelated to what I used to be looking out google for, but it was once indexed at the first page. I suppose your doing one thing right if Google likes you sufficient to put you at the first page of a non comparable search.

  24. 24 Hip Hop Blog said at 6:29 am on January 23rd, 2011:

    Bizarre this submit is totaly unrelated to what I was searching google for, but it used to be listed at the first page. I assume your doing one thing right if Google likes you enough to place you on the first web page of a non related search.

  25. 25 New York Bankruptcy Lawyer said at 10:49 am on January 24th, 2011:

    Thank you for making the honest effort to discuss this. I feel very robust approximately it and wish to learn more. If it’s OK, as you gain more extensive knowledge, could you mind adding more articles similar to this one with more information? It might be extremely helpful and useful for me and my friends.

  26. 26 hosting discount said at 3:36 pm on January 24th, 2011:

    Exactly what I was looking for, regards for putting up.

  27. 27 generic propecia said at 10:35 pm on January 24th, 2011:

    Quite good question

  28. 28 halo reach ranks said at 2:24 am on January 25th, 2011:

    I’m speechless. It is a very good weblog and very enticing too. Nice work! That’s not in reality so much coming from an newbie publisher like me, but it surely’s all I may just say after diving into your posts. Great grammar and vocabulary. Not like different blogs. You really recognise what you?re speaking about too. So much that you made me wish to discover more. Your weblog has change into a stepping stone for me, my friend.

  29. 29 lanyards said at 6:13 am on January 25th, 2011:

    I think that is an interesting point, it made me assume a bit. Thanks for sparking my pondering cap. Every so often I get such a lot in a rut that I just really feel like a record.

  30. 30 4 aces tobacco said at 9:29 am on January 25th, 2011:

    Excellent issues?I might observe that as someone who truly doesn’t write on blogs a lot (in reality, this can be my first post), I don’t suppose the time period ‘lurker’ may be very becoming to a non-posting reader. It’s now not your fault in the least , but perhaps the blogosphere could get a hold of a better, non-creepy title for the 90% people that revel in reading the content .

  31. 31 home education said at 12:36 am on January 26th, 2011:

    Have you noticed the news has changed its approach recently? What once seemed like a never discussed issue has become more prevelant. Its that time to chagnge our stance on this though.

  32. 32 baseball trading pins said at 1:34 am on January 27th, 2011:

    I like the valuable information you be offering to your articles. I can bookmark your weblog and have my youngsters check up here generally. I am relatively sure they’re going to be informed a number of new stuff here than anybody else!

  33. 33 english tutor said at 3:38 am on January 27th, 2011:

    i bookmarked this! looking for updates

  34. 34 electronic cigarette menthol said at 4:02 am on January 27th, 2011:

    I think that may be a fascinating element, it made me think a bit. Thanks for sparking my considering cap. Occasionally I am getting such a lot in a rut that I simply really feel like a record.

  35. 35 english tuition said at 11:16 am on January 27th, 2011:

    The one variation concerning him and Hitler is his frizzy incharge.

  36. 36 customized dog tag said at 1:55 am on January 28th, 2011:

    I am speechless. This is a very good blog and really attractive too. Nice work! That’s no longer in point of fact much coming from an beginner publisher like me, but it surely’s all I may say after diving into your posts. Great grammar and vocabulary. Now not like different blogs. You in point of fact understand what you?re talking about too. So much that you simply made me want to explore more. Your blog has turn out to be a stepping stone for me, my friend.

  37. 37 xenical said at 10:55 am on January 28th, 2011:

    All you need is love

  38. 38 health benefits of acai berry said at 12:02 am on February 1st, 2011:

    Great concept. Another informative post. This is a very nice blog that I will definitively come back to several more times this year!

  39. 39 webblog said at 3:40 am on February 2nd, 2011:

    This is such a great useful resource that you will be giving and you provide it away for absolutely free. I like finding sites which comprehend the value of supplying a high quality resource for free.

  40. 40 kitchen remodel ideas said at 10:30 am on February 2nd, 2011:

    I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you post.

  41. 41 NYC Movers said at 12:57 pm on February 2nd, 2011:

    City move can be and should be an easy process. With Manhattan Movers NYC it start with call to our office! No hidden costs, no surprices! Call us. We serve Manhattan, Brooklyn, Queens – all NYC. Manhattan Movers NYC, 553 Broome St, New York, NY 10013, (212)300-6628

  42. 42 Consumer Wealth System said at 10:58 pm on February 2nd, 2011:

    This is a very intriguing post, I was looking for this knowledge. Just so you know I found your web site when I was searching for blogs like mine, so please check out my site sometime and leave me a comment to let me know what you think.

  43. 43 mrscabies302 said at 3:00 am on February 4th, 2011:

    word available to be able to individuals in relation to your products or services,

  44. 44 New Zealand QROPS said at 12:00 pm on February 4th, 2011:

    Thanks for making the sincere effort to discuss this. I feel very robust approximately it and want to read more. If it’s OK, as you achieve more extensive wisdom, could you thoughts adding more articles very similar to this one with more information? It will be extraordinarily useful and helpful for me and my friends.

  45. 45 lifecell skin care said at 4:58 pm on February 4th, 2011:

    You completed certain fine points there. I did a search on the theme and found most persons will have the same opinion with your blog.

  46. 46 wash lights said at 11:22 pm on February 4th, 2011:

    I’m speechless. This is a superb weblog and really engaging too. Great paintings! That’s no longer actually much coming from an newbie publisher like me, but it surely’s all I may just say after diving into your posts. Great grammar and vocabulary. No longer like different blogs. You really realize what you?re talking approximately too. Such a lot that you made me want to discover more. Your weblog has turn into a stepping stone for me, my friend.

  47. 47 lincoln radio said at 10:32 am on February 5th, 2011:

    I picture this could be various upon the written content material? even so I nonetheless consider that it usually is appropriate for virtually any type of matter subject matter, as a result of it could frequently be gratifying to resolve a heat and pleasant face or maybe pay attention a voice while initial landing.

  48. 48 Natural weight loss pills said at 5:12 am on February 6th, 2011:

    Why there are so many comments in this blog?

  49. 49 ladies watches said at 7:38 pm on February 6th, 2011:

    Once you have a home theater system to see films, you must fully bundle this out by having a good surround device that offers you the greatest audio possible

  50. 50 Frauen kennenlernen said at 10:01 pm on February 6th, 2011:

    Looking forward to another great article. Good luck to the author! all the best!

  51. 51 blackberry 8900 curve said at 11:32 pm on February 6th, 2011:

    This is a very intriguing post, I was looking for this knowledge. Just so you know I found your web site when I was searching for blogs like mine, so please check out my site sometime and leave me a comment to let me know what you think.

  52. 52 porno izle said at 12:13 am on February 7th, 2011:

    Your site is very slow due to open

  53. 53 Fab Sri Lanka said at 2:24 am on February 7th, 2011:

    I actually love how it is easy on my eyes and also the information are well written. I am wondering how I can be notified whenever a new post has been made. I have subscribed to your rss feed which ought to do the trick!

  54. 54 buy neopoints said at 4:08 am on February 7th, 2011:

    Thanks for the nice blog. It was very useful for me. Keep sharing such ideas in the future as well. This was actually what I was looking for, and I am glad to came here! Thanks for sharing the such information with us.

  55. 55 fashion jewellery said at 4:14 am on February 7th, 2011:

    Anybody knows anything about island Korcula beaches?

  56. 56 fatburners said at 6:02 am on February 7th, 2011:

    Can I just say what a aid to seek out someone who actually is aware of what theyre speaking about on the internet. You definitely know how you can carry a difficulty to mild and make it important. More individuals must read this and understand this facet of the story. I cant imagine youre not more common because you definitely have the gift.

  57. 57 lanyards said at 10:54 pm on February 7th, 2011:

    Thanks for making the sincere attempt to speak about this. I think very robust approximately it and want to read more. If it’s OK, as you achieve extra intensive wisdom, may you thoughts adding more articles very similar to this one with additional info? It would be extremely useful and useful for me and my friends.

  58. 58 phen375 diet pills said at 10:42 am on February 8th, 2011:

    Spot on with this write-up, I really assume this website wants way more consideration. I’ll in all probability be once more to read much more, thanks for that info.

  59. 59 my site said at 10:58 am on February 8th, 2011:

    I admire the dear knowledge you offer on your articles. I can bookmark your weblog and feature my kids check up here generally. I’m reasonably sure they’ll learn loads of new stuff here than any one else!

  60. 60 Rihanna said at 4:46 pm on February 8th, 2011:

    Good write-up, I am a major time fan of the blog, continue to keep up the Very well written work, and I will probably be a ordinary visitor for a pretty long time.

  61. 61 Dell said at 5:58 pm on February 8th, 2011:

    I advise this site to my acquaintances so it could possibly be valuable & informative for them also. Ideal effort.

  62. 62 Dell said at 6:40 pm on February 8th, 2011:

    This article has lot of helpful and valuable information related to the topic and is also very nicely written. I really wish to see such more posts in future. Excellent posting!

  63. 63 Rihanna said at 7:15 pm on February 8th, 2011:

    I genuinely enjoyed this. It was very educational and helpful. I’ll return to check on upcoming posts

  64. 64 Dickson TN said at 2:36 am on February 9th, 2011:

    I actually love how it is easy on my eyes and also the information are well written. I am wondering how I can be notified whenever a new post has been made. I have subscribed to your rss feed which ought to do the trick!

  65. 65 Elvis said at 2:44 am on February 9th, 2011:

    Well , the view of the passage is totally correct ,your details is really reasonable and you guy give us valuable informative post

  66. 66 funny facebook status said at 5:41 am on February 9th, 2011:

    Unusual this publish is totaly unrelated to what I used to be searching google for, nevertheless it was indexed at the first page. I assume your doing something right if Google likes you sufficient to position you at the first page of a non related search.

  67. 67 Pioneer vsx-1020-k review said at 9:22 am on February 9th, 2011:

    I feel that is an enchanting aspect, it made me assume a bit. Thanks for sparking my considering cap. Now and again I get such a lot in a rut that I just really feel like a record.

  68. 68 Bali Reseguide said at 6:51 am on February 10th, 2011:

    really enjoyed what you have written actually. it really is not that simple to discover even remotely good posts to read (you know READ and not just browsing through it like some uniterested and flesh eating zombie before going somewhere else), so cheers mate for not wasting any of my time! :p

  69. 69 vyvoz stroitelnogo musora said at 2:16 pm on February 10th, 2011:

    vyvoz stroitelnogo musora spb…

    vyvoz musora…

  70. 70 PPI said at 9:22 pm on February 10th, 2011:

    I am very impress your this post, this is too much interesting. Thanks a lot…

  71. 71 dfehgiuoe said at 9:46 pm on February 10th, 2011:

    Such a great blog. I spend days on the internet reading blogs, about tons of different subjects. I must first of all give kudos to whoever created your theme and second of all for you for writing things i can only call a fantastic post.

  72. 72 paketresa said at 10:26 pm on February 10th, 2011:

    kinda appreciated what you wrote actually. it just is not that easy to find great text toactually read (you know READ and not simply browsing through it like a zombie before going somewhere else), so cheers man for not wasting any of my time! :)

  73. 73 lower blood pressure naturally said at 6:30 am on February 11th, 2011:

    So informative site! Big thanks! Thanks for a great time visiting your site. It’s really a pleasure knowing a site like this packed with great information. Thank you!

  74. 74 portable cell jammer said at 11:59 am on February 11th, 2011:

    I believe that may be an enchanting element, it made me suppose a bit. Thank you for sparking my considering cap. Every so often I get so much in a rut that I just really feel like a record.

  75. 75 Dell said at 12:03 pm on February 11th, 2011:

    I honestly like your web log material the way in which you put up the things…I’ve learn the matter with extraordinary interest.

  76. 76 Justin Bieber said at 3:47 pm on February 11th, 2011:

    i like this site.! have a lot of information.!!!.. these type of sites always magnetize me to submit comment…

  77. 77 Rihanna said at 4:21 pm on February 11th, 2011:

    I really loved reading your blog. It was tremendously properly authored and easy to undertand. Unlike other blogs I have read which are really not that fine. I also found your posts fairly interesting. In fact after reading, I had to go show it to my friend and he enjoyed it as effectively!

  78. 78 May Estain said at 2:44 am on February 12th, 2011:

    Hello, probably i¡¯m being a little off topic here, but I was browsing your site and it looks stimulating. I¡¯m writing a blog and trying to make it look cool, but everytime I touch it I mess something up. Did you design the blog yourself? Could someone with little experience do it, and add updates without messing it up? Anyways, good info on here, very informative.

  79. 79 patches said at 5:00 am on February 12th, 2011:

    you may have an awesome weblog right here! would you prefer to make some invite posts on my weblog?

  80. 80 sc1 said at 11:39 am on February 12th, 2011:

    Can I just say what a relief to seek out somebody who truly knows what theyre speaking about on the internet. You undoubtedly know how you can deliver a problem to light and make it important. Extra people must read this and perceive this side of the story. I cant imagine youre no more in style because you undoubtedly have the gift.

  81. 81 the worlds hardest riddle with answer said at 6:04 pm on February 13th, 2011:

    gordon91DDE66D@fireden.net jarred91DDE66D@fireden.net All you need to discern around computer tech support With the abundance of online computer tech supporter military talents providers

  82. 82 funny dogs said at 8:50 pm on February 13th, 2011:

    keep up the good work on the site. Do like it! :p Could maybe use some more updates more often, but i am sure you got better or other stuff to do , hehe. =p

  83. 83 short-term loans said at 2:53 am on February 14th, 2011:

    Good post. I study one thing tougher on different blogs everyday. It will at all times be stimulating to read content from other writers and practice a bit of one thing from their store. I’d prefer to make use of some with the content material on my weblog whether you don’t mind. Natually I’ll provide you with a hyperlink on your internet blog. Thanks for sharing.

  84. 84 zero friction marketing said at 7:14 pm on February 14th, 2011:

    You made certain nice points there. I did a search on the subject and found nearly all persons will go along with with your blog.

  85. 85 tory-burch-flip-flops said at 11:04 pm on February 14th, 2011:

    discount tory burch shoes…

    Tory Burch flip flop online sale.For the younger working expat women,look at these tory burch Shoes.these tory burch flip flops are definitely fun,yet the glossy design and round-toe stay these lovely.These tory burch shoes with multiple.New arrivel To…

  86. 86 Vigrx Plus pill said at 10:31 am on February 15th, 2011:

    Excellent points?I’d observe that as somebody who actually doesn’t write on blogs much (if truth be told, this may be my first publish), I don’t assume the time period ‘lurker’ is very becoming to a non-posting reader. It’s no longer your fault in the slightest degree , but most likely the blogosphere may just get a hold of a greater, non-creepy title for the ninety% of us that enjoy reading the content material .

  87. 87 375phen said at 4:47 pm on February 15th, 2011:

    WONDERFUL Post.thanks for share..extra wait .. …

  88. 88 Paketresorna said at 11:21 pm on February 16th, 2011:

    kinda liked what you posted actually. it just isn’t that simple to find good text to read (you know really READ and not simply browsing through it like a zombie before going to yet another post to just ignore), so cheers mate for really not wasting my time on the god forsaken internet. :D

  89. 89 gordonii said at 4:37 am on February 17th, 2011:

    Aw, this was a very nice post. In idea I wish to put in writing like this additionally – taking time and actual effort to make a very good article… but what can I say… I procrastinate alot and on no account appear to get something done.

  90. 90 Gregory Despain said at 3:00 pm on February 17th, 2011:

    I love your design, where can I get it? please let me know. My email: en533@hotmail.com

  91. 91 oem cheap software said at 8:02 pm on February 17th, 2011:

    Those who do not learn from history are doomed to repeat it

  92. 92 meratol tablets said at 2:34 am on February 18th, 2011:

    A formidable share, I just given this onto a colleague who was doing slightly analysis on this. And he in reality purchased me breakfast as a result of I found it for him.. smile. So let me reword that: Thnx for the treat! However yeah Thnkx for spending the time to discuss this, I feel strongly about it and love reading more on this topic. If doable, as you develop into expertise, would you mind updating your blog with more details? It is extremely helpful for me. Huge thumb up for this blog submit!

  93. 93 web hosting delhi said at 3:09 am on February 18th, 2011:

    Thank you for making the honest strive to provide an explanation for this. I feel very strong approximately it and wish to learn more. If it’s OK, as you attain extra extensive knowledge, could you mind adding more posts very similar to this one with additional information? It could be extremely useful and helpful for me and my colleagues.

  94. 94 payday advance said at 8:48 am on February 18th, 2011:

    It’s arduous to find educated individuals on this subject, but you sound like you know what you’re speaking about! Thanks

  95. 95 praca wroclaw said at 5:41 am on February 21st, 2011:

    Not everything that is faced can be changed, but nothing can be changed until it is faced. ~James Baldwin

  96. 96 mental health said at 10:09 am on February 21st, 2011:

    Phyllis Bottome~ There are two ways of meeting difficulties You alter the difficulties or you alter yourself to meet them.