I’ve been playing with Erlang over nights and weekends off and on for a few months now. I’m loving it for several reasons. First off, it’s completely different than any other programming language I’ve worked with. It makes me think rather than take things for granted. I’m intrigued by concurrency abilities and its immutable no-shared-state mentality. I do go through periods of amazing productivity followed by hours if not days of figuring out a simple task. Luckily the folks in #erlang on irc.freenode.net have been extremely patient with myself and the hundreds of other newcomers and have been extremely helpful. I’m also finding that those long pain periods are happening less frequently the longer I stick with it.
One of the things that truly blew me away about Erlang (after the original Erlang Now! moment) is its bit syntax. The bit syntax as documented at erlang.org and covered in Programming Erlang is extremely powerful. Some of the examples in Joe’s book such as parsing an MP3 file or an IPv4 datagram hint at the power and conciseness of binary matching and Erlang’s bit syntax. I wanted to highlight a few more that have impressed me while I was working on some network socket programming in Erlang.
There are several mind-boggling examples in Applications, Implementation and Performance Evaluation of Bit Stream Programming in Erlang (PDF) by Per Gustafsson and Konstantinos Sagonas. Here are two functions for uuencoding and uudecoding a binary:
uuencode(BitStr) -> << (X+32):8 || <<X:6>> <= BitStr >>. uudecode(Text) -> << (X-32):6 || <<X:8>> <= Text >>.
UUencoding and UUdecoding isn’t particularly hard, but I’ve never seen an implementation so concise. I’ve also found that Erlang’s bit syntax makes socket programming extremely easy. The gen_tcp library makes connection to TCP sockets easy, and Erlang’s bit syntax makes creating requests and processing responses dead simple too.
Here’s an example from qrbgerl, a quick project of mine that receives random numbers from the Quantum Random Bit Generator service. The only documentation I needed to use the protocol was the Python client and the C++ client. Having access to an existing Python client helped me bridge the “how might I do this in Python?” and “how might I do this in Erlang?” gaps, but I ended up referring to the canonical C++ implementation quite a bit too.
I start out by opening a socket and sending a request. Here’s the binary representation of the request I’m sending:
list_to_binary([<<0:8,ContentLength:16,UsernameLength:8>>, Username, <<PasswordLength:8>>, Password, <<?REQUEST_SIZE:32>>]),
This creates a binary packet that complies exactly with what the QRBG service expects. The first thing that it expects is a 0 represented in 8 bits. Then it wants the length of the username plus the length of the password plus 6 (ContentLength above) represented in 16 bits. Then we have the username represented as 8 bit characters/integers, followed by the length of the password represented in 8 bits, followed by the password (again as 8 bit characters/integers). Finally we represent the size of the request in 32 bits. In this case the macro ?REQUEST_SIZE is 4096.
While that’s a little tricky, once we’ve sent the request, we can use Erlang’s pattern matching and bit syntax to process the response:
<<Response:8, Reason:8, Length:32, Data:Length/binary, _Rest/binary>> = Bin,
We’re matching several things here. The response code is the first 8 bits of the response. In a successful response we’ll get a 0. The next 8 bits represent the reason code, again 0 in this case. The next 32 bits will represent the length of the data we’re being sent back. It should be 4096 bytes of data, but we can’t be sure. Next we’re using the length of the data that we just determined to match that length of data as a binary. Finally we match anything else after the data and discard it. This is crucial because binaries are often padded at the beginning or end of the stream. In this case there’s some padding at the end that we need to match but can safely discard.
Now that we have 4096 bytes of random bits, let’s do something with them! I’ve mirrored the C++ and Python APIs as well as I could, but because of Erlang’s no shared state it’s going to look a little different. Let’s match a 32 bit integer from the random data that we’ve obtained:
<<Int:32/integer-signed, Rest/binary>> = Bin,
We’re matching the first 32 bits of our binary stream to a signed integer. We’re also matching the rest of the data in binary form so that we can reuse it later. Here’s that data extraction in action:
5> {Int, RestData} = qrbg:extract_int(Data).
{-427507221,
<<0,254,163,8,239,180,51,164,169,160,170,248,94,132,220,79,234,4,117,
248,174,59,167,49,165,170,154,...>>}
6> Int.
-427507221
I’ve been quite happy with my experimentation with Erlang, but I’m definitely still learning some basic syntax and have only begun to play with concurrency. If the above examples confuse you, it might help to view them in context or take a look at the project on google code. I have also released an ISBN-10 and ISBN-13 validation and conversion library for Erlang which was a project I used to teach myself some Erlang basics. I definitely have some polishing to do with the QRBG client, but isbn.erl has full documentation and some 44 tests.
Comments
74 responses to “Erlang bit syntax and network programming”
Unfortunately bit comprehensions and expressions such as > are not yet part of the language (as of R11B-5). But they would surely be very useful if they ever get included.
Html filtering problem: what I meant to say was expressions such as dynamic field length in binaries.
[…] Erlang bit syntax and network programming at Matt Croydon::Postneo (tags: bit erlang network programming socket) […]
They are in r11b5 just not documented, and not enabled by default, they are enabled and documented in r12b
Прошу прощениÑ, ничем не могу помочь, но уверен, что Вам обÑзательно помогут. Ðе отчаивайтеÑÑŒ.
Ðомер не пройдет!
This blog appears to recieve a large ammount of visitors. How do you promote it? It offers a nice individual twist on things. I guess having something real or substantial to post about is the most important thing.
Where can work crews discover low priced interest groups?
Thanks for putting this up, it was quite handy and told me very much
I put your blog to my favorites. And I will definitely I visit your site more often! Stay sharp and keep up the terrific work 🙂
very good \o/
Look high, look low, and we see that gamblers actually form the majority of the world’s inhabitants. – James Runciman, Side Lights (1893)
Whenever I think of the past, it brings back so many memories…
Don’t think it, ink it. -Mark Victor Hansen
That said…
While browsing through yahoo and google for better useful resource to help for the topic, this is by far the most effective one i have found. Thank you
I appreciate for this perfect matter and narrative of current interrogation. You’ve laid it out greater than any discussion I have seen. Also thx for citing my text on it. Your’s takes it way greater.
Immediately turn your living room to a theatre with a projecter, big screen together with a complete sound system which has surround sound
For people who have a home movie theater system to look at films, you should definitely completely packet this out with a good surround system that offers you the best sound attainable
great thanks \o/
very good \o/
Jacksonville beach real estate information to be found here. If your in Northeast Florida and in need of a new home, check out this website right here. Thank you
I love the valuable knowledge you offer for your articles. I will be able to bookmark your weblog and have my kids take a look at up here generally. I’m slightly positive they’re going to be informed a variety of new stuff right here than anybody else!
Stumbled across your post while searching via yahoo. I understand the first paragraph and its excellent! I do not have time to finish it now, but I have bookmarked your website and will examine the rest tonight. : )
I picture this may well be numerous upon the written content material? in spite of this I still imagine that it could be appropriate for just about any type of subject subject material, as a result of it would incessantly be pleasant to determine a warm and pleasant face or possibly pay attention a voice while preliminary landing.
I picture this might be various upon the written content? however I still imagine that it can be suitable for virtually any type of matter material, as a result of it could steadily be gratifying to resolve a warm and delightful face or maybe pay attention a voice whilst preliminary landing.
I admire the precious knowledge you be offering in your articles. I will bookmark your blog and feature my kids take a look at up right here generally. I am quite sure they’re going to be informed loads of new stuff right here than anyone else!
Thanks for making the sincere effort to talk about this. I believe very strong about it and would like to read more. If it’s OK, as you acquire extra extensive wisdom, could you mind adding extra articles similar to this one with more information? It might be extremely useful and helpful for me and my friends.
Howdy, I just hopped over to your website online by way of StumbleUpon. Not one thing I’d typically learn, however I favored your thoughts none the less. Thank you for making one thing price reading.
Hello, I just hopped over in your website by way of StumbleUpon. Not something I would typically learn, but I liked your feelings none the less. Thanks for making one thing price reading.
I believe that may be a fascinating element, it made me assume a bit. Thank you for sparking my pondering cap. Every so often I am getting such a lot in a rut that I simply feel like a record.
Rather valuable phrase
I believe that is an interesting point, it made me think a bit. Thanks for sparking my pondering cap. Infrequently I get such a lot in a rut that I just feel like a record.
I image this might be various upon the written content? nonetheless I nonetheless consider that it may be appropriate for just about any type of topic subject matter, as a result of it will often be enjoyable to decide a heat and delightful face or possibly hear a voice whilst initial landing.
Good points?I would observe that as anyone who really doesn’t write on blogs so much (actually, this can be my first publish), I don’t assume the term ‘lurker’ may be very becoming to a non-posting reader. It’s now not your fault the least bit , however in all probability the blogosphere may get a hold of a better, non-creepy identify for the 90% of us that experience studying the content .
Thanks! Those dudes at your competition (I don’t need to say who) don’t even have a clue! Keep it up! I have a health site of my own. I will place a link back to your post. Thanks!
Odd this submit is totaly unrelated to what I used to be looking out google for, but it used to be listed at the first page. I assume your doing one thing right if Google likes you enough to put you at the first page of a non related search.
I picture this might be various upon the written content material? having said that I still consider that it may be suitable for nearly any type of subject subject matter, because it could incessantly be pleasing to resolve a warm and delightful face or maybe listen a voice when preliminary landing.
thanks for doing the research, and sharing!
This was a great post,. I was surfing a found a few other sites that seem to be similar Chicago Flat Fee MLS and here Chicago FSBO
Thanks for posting, is anyone else using chrome having a problem loading the whole page?
Thanks for making the honest try to provide an explanation for this. I believe very sturdy approximately it and want to be informed more. If it’s OK, as you reach more intensive wisdom, would you thoughts including more posts similar to this one with more information? It might be extraordinarily helpful and helpful for me and my colleagues.
I picture this could be diverse upon the written content material? however I still believe that it could be suitable for virtually any type of subject material, as a result of it would frequently be pleasing to determine a warm and pleasant face or perhaps listen a voice when initial landing.
City move can be and should be an easy process. With Manhattan Movers NYC it start with call to our office! No hidden costs, no surprices! Call us. We serve Manhattan, Brooklyn, Queens – all NYC. Manhattan Movers NYC, 553 Broome St, New York, NY 10013, (212)300-6628
Thanks for making the trustworthy effort to discuss this. I believe very robust about it and wish to read more. If it’s OK, as you acquire more extensive knowledge, would you thoughts adding more articles similar to this one with more information? It could be extremely helpful and useful for me and my friends.
Please let me know if you’re looking for a writer for your blog. You have some really good posts and I believe I would be a good asset. If you ever want to take some of the load off, I’d really like to write some articles for your blog in exchange for a link back to mine. Please shoot me an email if interested. Regards!
I picture this could be diverse upon the written content? nevertheless I nonetheless imagine that it usually is appropriate for nearly any form of matter material, because it will regularly be fulfilling to determine a warm and delightful face or possibly listen a voice whilst preliminary landing.
Great reading!I also share some other ideias in my blog
ПроÑто Ñ…Ð¾Ñ€Ð¾ÑˆÐ°Ñ Ñтраничка
Atypical this submit is totaly unrelated to what I was searching google for, nevertheless it was once indexed on the first page. I suppose your doing something proper if Google likes you enough to position you on the first web page of a non related search.