LinuxCon. Day III. Final.

The third(and the last) day for me can be summarised pretty easily by “I saw Linus!”. :) Indeed, I saw Linus, although I almost missed his keynote. I got kinda late(10 minutes before the start) to the room where the talk was supposed to take place and the organisers weren’t letting anyone in despite lots of chairs being free inside. I believe that it had something to do with german buerocracy and they were following some strange process. But before riots started, some woman from US came in and put pressure on local organizers to let us in. To our joy they gave up and let us in, right before the Linus’ talk(Note to self: come earlier next time).Linus Continue reading

Still polishing my gem

It took me a while to realise why I was seeing some upload speed drops in sanguinews. And when I did realise why it was happening(because of inefficient crc32 calculation), it took me quite some time to restructure internal logic in the code. But now, I am quite satisfied with the result.

Version 0.70 is fast, stable and feature rich. I think that I’ll switch version number to 1.0.0 after implementing header check server. I was able to reach 9MB/s(no SSL) and 6.5MB/s(with SSL) on my low-end test machine. I think more isn’t even physically possible on it. So, don’t hesitate to upgrade. It’s definitely worth it!

Faster, faster, better…

And here comes… one more performance optimization update! I mentioned this already, but I’ll repeat myself:

Zlib’s CRC32 calculation is suboptimal!

Yes, it has its uses. It can be used for byte-by-byte calculations, but there are at least 3 algorithms which show better performance:

I couldn’t find any implementation of these algorithms for ruby and I wanted to give sanguinews’ users the best performance possible, so I wrote crc32 gem specifically for this task. It uses Intel’s slicing-by-8 algorithm. In pure CRC32 calculations it shows about 4 times better performance than standard Zlib library. But in sanguinews we are dealing with files and effective performance will be limited by your IO throughput. Still, it should give you about two times better performance in CRC32 calculations.

Of course, there are some binary uploaders out there, which do not care about correct CRC32 checksum, considering that it’s not mandatory by yenc standard, however it’s still recommended and that’s why I want to have it in my tool.

New crc32 gem is used in sanguinews starting from version 0.63, which can be downloaded already from rubygems.org.

Encode faster with newest version

I know that I told in comments that I dislike coding in C. Nothing has changed. I really do, but nonetheless I enjoy optimising code as much as it’s possible. That’s why I liked coding in assembler in university and that’s why I felt motivated enough to delve into theory of CRC32 calculation. So, the newest update brings less IO overhead and less CPU cycles for the whole yencoding process. It will help you achieve better speed when uploading(still limited to your usual network throughput though).

Nothing groundbreaking. About 10%, but I am planning to improve performance even more by getting rid of Zlib’s CRC32 algorithm completely. I am still using it to calculate CRC32 for the whole file. It is inefficient in this case. So, it needs to be changed. It means more C code.  :(

For the time being, you can download the latest version from rubygems.org or get it with gem install sanguinews command.

Lotsa fun today

I had a possibility today to test ride some bikes. There were many options, but I eventually chose 3 models:

Honda was somehow very sensitive on low gears. Slightest turn of my wrist made me shiver a little, because it felt soooo… unpredictable. Maybe you just need to get used to it, dunno.  Anyway, this jerky acceleration wasn’t as bad compared to my experience on the second bike – KTM Duke.

I have never ridden supermoto before so every turn felt like a big challenge. I even tried to stick out my leg, like motocross riders do, but it felt completely wrong nonetheless.

Yamaha R6The last one, Yamaha R6, was a great surprise. Yes, sitting position isn’t made for long tours, but despite this everything else was outstanding. Easy to turn, smooth to accelerate, enough power to satisfy almost everyone. Very, very nice bike. I had a blast with it and I definitely want it now…

Big update(v0.60)

Ok, maybe it isn’t so big from a user’s perspective, but from developers perspective this is a huge update. I restructured almost everything, reshuffled methods, created a proper gem…
Apropos, now you don’t need to pull the code. It’s still a nice thing to do if you want to create a patch, but it isn’t required for a user. You don’t need to execute bundle commands either. Just run:

gem install sanguinews

…and enjoy!

P. S. After receiving some feedback, I am planning to implement the support for a secondary server now, just for header checking purposes. This is aimed at those of you, who have 2 accounts: a regular and block one. This will save your bandwidth, when you are uploading with your block account’s credentials.

Header checking is here!

ruby logoThe second most awaited feature is here! In the most recent release! Yes, I am talking about header checking. After code refactoring in version 0.50, it wasn’t so hard to introduce this functionality.

To start using it, you only need to update your sanguinews release to version 0.54 and specify -C or --check option.

In case you are interested in technical details, header checking we’ll be performed by the faster STAT command. My early tests showed quite good results. Hopefully completion rate will be satisfactory now even on the most busiest servers.

More updates

And here comes the version 0.53 with the most asked for feature – progress bar. Along with some bugfixes introduced in the two previous versions. The next thing on the improvement list would be header checking. That’s exactly what I’m planning to implement soon… Ok, I don’t want to jinx it, but I really don’t expect it to take a year this time. :)
Got some more ideas or suggestions? Don’t hesitate to post them!

The Way of Tad