Getting back to development isn’t easy. Some parts of code, which seamed logical few months ago, now look strange. You need to recall what some functions do and why you did put them there. It’s tough. It’s almost like reading someone else’s code. Nonetheless it’s still fun. Otherwise I wouldn’t do it.
I continue to go through parts I didn’t like before and improve them. This time I introduced a way to queue file encoding jobs. Before encoding was being done sequentially – one file after another. With fast enough computer with a good upload speed it resulted in spikes on a network speed graph. Not pretty.
Version 0.80 fixes this behaviour and improves upload speed a bit as a result. So, if you are running an older version, it’s time to upgrade!
The last few month were kinda busy for me. Lots of travel, lots of training, lots of new challenges. Only now I’m getting back to, sadly slowly, sanguinews’ development.
Thanks to your feedback, unexpected behaviour was fixed. On some systems sanguinews wasn’t working. Definitely a nasty bug.
Starting version 0.71 files will be also uploaded in alphabetical order(sorting was system dependant before; sometimes files were sorted in one order, sometimes in another). This closes issue #27.
That’s it. As I said, nothing major, nothing game breaking, but the app just got better nonetheless!
It took me a while to realise why I was seeing some upload speed drops in sanguinews. And when I did realise why it was happening(because of inefficient crc32 calculation), it took me quite some time to restructure internal logic in the code. But now, I am quite satisfied with the result.
Version 0.70 is fast, stable and feature rich. I think that I’ll switch version number to 1.0.0 after implementing header check server. I was able to reach 9MB/s(no SSL) and 6.5MB/s(with SSL) on my low-end test machine. I think more isn’t even physically possible on it. So, don’t hesitate to upgrade. It’s definitely worth it!
And here comes… one more performance optimization update! I mentioned this already, but I’ll repeat myself:
Zlib’s CRC32 calculation is suboptimal!
Yes, it has its uses. It can be used for byte-by-byte calculations, but there are at least 3 algorithms which show better performance:
I couldn’t find any implementation of these algorithms for ruby and I wanted to give sanguinews’ users the best performance possible, so I wrote crc32 gem specifically for this task. It uses Intel’s slicing-by-8 algorithm. In pure CRC32 calculations it shows about 4 times better performance than standard Zlib library. But in sanguinews we are dealing with files and effective performance will be limited by your IO throughput. Still, it should give you about two times better performance in CRC32 calculations.
Of course, there are some binary uploaders out there, which do not care about correct CRC32 checksum, considering that it’s not mandatory by yenc standard, however it’s still recommended and that’s why I want to have it in my tool.
New crc32 gem is used in sanguinews starting from version 0.63, which can be downloaded already from rubygems.org.
I know that I told in comments that I dislike coding in C. Nothing has changed. I really do, but nonetheless I enjoy optimising code as much as it’s possible. That’s why I liked coding in assembler in university and that’s why I felt motivated enough to delve into theory of CRC32 calculation. So, the newest update brings less IO overhead and less CPU cycles for the whole yencoding process. It will help you achieve better speed when uploading(still limited to your usual network throughput though).
Nothing groundbreaking. About 10%, but I am planning to improve performance even more by getting rid of Zlib’s CRC32 algorithm completely. I am still using it to calculate CRC32 for the whole file. It is inefficient in this case. So, it needs to be changed. It means more C code. 🙁
For the time being, you can download the latest version from rubygems.org or get it with
gem install sanguinews command.
Ok, maybe it isn’t so big from a user’s perspective, but from developers perspective this is a huge update. I restructured almost everything, reshuffled methods, created a proper gem…
Apropos, now you don’t need to pull the code. It’s still a nice thing to do if you want to create a patch, but it isn’t required for a user. You don’t need to execute
bundle commands either. Just run:
gem install sanguinews
P. S. After receiving some feedback, I am planning to implement the support for a secondary server now, just for header checking purposes. This is aimed at those of you, who have 2 accounts: a regular and block one. This will save your bandwidth, when you are uploading with your block account’s credentials.
The second most awaited feature is here! In the most recent release! Yes, I am talking about header checking. After code refactoring in version 0.50, it wasn’t so hard to introduce this functionality.
To start using it, you only need to update your sanguinews release to version 0.54 and specify
In case you are interested in technical details, header checking we’ll be performed by the faster STAT command. My early tests showed quite good results. Hopefully completion rate will be satisfactory now even on the most busiest servers.
And here comes the version 0.53 with the most asked for feature – progress bar. Along with some bugfixes introduced in the two previous versions. The next thing on the improvement list would be header checking. That’s exactly what I’m planning to implement soon… Ok, I don’t want to jinx it, but I really don’t expect it to take a year this time. 🙂
Got some more ideas or suggestions? Don’t hesitate to post them!
Yes, it’s official. The new version of sanguinews is here. It took me almost a year to regain the interest and find some time to fix everything what was bothering me. I don’t know if it would have happened, if not the feedback I was getting here and on the github. It’s really nice to see that someone uses my applications.
I wasn’t been able to test the code as extensively as I would like to, but from the looks of it, everything should be much more stable. I rewrote so much of internal logic that I can only say that the difference between 0.48 and 0.50 is really huge. Try it!
Now I need to fix another imperfect part of the code – upload status notification…
Yesterday I’ve released sanguinews v0.47 and this release marks the end of alpha stage. It doesn’t mean that the software is bug free. It means that I am quite happy with it and it should be able to provide desired results. If there are some bugs(and I presume that there are) – I don’t know about them at the moment. So, I will need your help. Use it, test it, inform me of any strange behavior. Bugs need to be crushed!
In the meantime, I will be improving the “progressbar” feature. Currently it displays only the average speed. I will be adding “ETA” and some kind of actual, moving progress/status bar. One more thing that I would like to see in my program is the capability to tweak headers. I am fan of Gentoo distribution. I like tinkering with the system and I think that a user should have as much freedom and options as it’s possible. Usenet’s standards include quite a lot of headers. Some are required, some are taken by usenet providers, but there are some that could be used, for any reason, by us. I will make sure that there is an easy way to change them.
What I will do after that? No idea. Time will tell, but at the moment I am still having fun and it means that I will continue improving this particular project of mine.