rarnpar

Introduction

Few years ago, when one my friend decided to try out Linux, he has noticed that there is no easy way to prepare files for an usenet upload. Although I barely knew what Usenet is, I decided to provide a simple solution – write a script, which will simplify the whole process as much as possible.

I still consider myself a Usenet-newbie and the friend, who was the reason I wrote the script, switched back to Windows, but “rarnpar” has received some recognition in a meantime and it was even mentioned on reddit(which boosted visits for my blog a lot). Because of all the feedback I was getting, I was improving the script, step by step, and now time has come to dedicate a page entirely for this simple rar & par2 wrapper.

If you find it useful, I would appreciate a comment or a suggestion how to improve it.

If you don’t know how to start, well, I have a “How to” guide prepared by my friend(who wishes to remain anonymous), which will(hopefully answer your questions. If you still have questions after reading it, I would try answering them, just, please keep in mind, that I don’t know, what “block size” is and I don’t know why some things were implemented the way they were – the script was written for other people to use. I haven’t had a need for it and I am doing only basic tests after I add some new features. Because of this there could be bugs. It’s in your power to help me fix them – just write a message, when you see that something isn’t right.


Tad-Do’s Rarnpar Mini-Guide(by anonymous)

Q&A:

1) What’s this?

  • Rarnpar is a script written specifically for making it easy to RAR and PAR2 lots of files\folders at once

2) System requirements?

  • It runs on Linux distros.

3) A monkey could do it!

  • Well,it is a scientific fact that we share common ancestors with monkeys. On a serious note, the script might be of use to Usenet uploaders, especially the ones who don’t know much about scripting because it offers advanced features such as:
    • a) Cryptorenaming mode
    • b) Exclusions
    • c) Simplified NFO creating
    • d) Mediainfo support
    • e) Folder\file mode
    • f) Prefix\suffix
    • g) SFV creation

4) Uh? I feel stupid!

  • No worries! Everything will be explained in time.

INSTALLATION

Disclaimer:

I presume that you are not a hardcore command line only linux geek (if it was the case, you wouldn’t be reading all this, would you?), so I assume that you do have desktop environment available on your OS.

Taking this into account I won’t be addressing how to do all this stuff via commands only.
1) Go to https://tad-do.net/wp-content/rarnpar,copy and paste into any text editor and save as yourfilename

2) Right mouse click on the file and set permissions to “Allow executing file as a program”

Alternatively run in a terminal opened in a folder where you saved the file “sudo chmod +x yourfilename” or “sudo chmod +x /folder1/folder2/../yourfilename

3) Copy\move the file to /usr/local/bin

You will need root access to do that.

If a terminal is opened in the file’s folder run “sudo cp yourfilename /usr/local/bin/” otherwise run “sudo cp /folder1/folder2/../yourfilename /usr/local/bin/
4) You will need some dependencies installed in order to run the script.

We will do it via a terminal

a)RAR

sudo apt-get install rar

b)PAR2

sudo apt-get install par2

c)SFV

sudo apt-get install cksfv

d)MediaiNFO

sudo apt-get install mediainfo

e)Coreutils

They are already installed on any linux distro.

To check if they are, run in terminal “dpkg -s coreutils

OPTIONS

Let’s assume that you named rarnpar file as rarnpar πŸ˜‰

Having put rarnpar into /usr/local/bin we can now run it anywhere we like from a terminal

If you run it without any options, then it automatically runs as “rarnpar --help” which in turn will show you all the options available and a brief explanation.

1)-b (default 384000)

Example: -b 640000

par2 block size. Default value is 384000.

Which equals 3000 lines (option that you have to set in your Usenet upload client).

Other popular values are:

640000 (=5000 lines) and 1280000(=10000 lines)

Hint: You shouldn’t make it larger than 1280000 and 10000 lines in your upload client because newsservers might reject articles you will be uploading with numbers larger than these.

2)-B

Par2 block count. You can’t use both -b and -B.

Hint: I suggest that you stick to -b.

3)-d

Creates a folder for each processed file\folder

4)-D

Specifies a working directory where files\folders you want to process are

Generally there is no need to do that if you run the script from a terminal opened in this folder

Example:-D /home/username/downloads/

5)-e

Comma separated list of expressions to exclude

Example: -e "*.avi","*.mpg"

6)-h or --help

Help ;P

7)-l

Limits size of recovery files, Don’t use both -l and -u)

Hint:Leave it

8)-m (default 0)

Rar compression level from 0 to 5

Hint: We don’t generally use compression for archives to be uploaded to Usenet or anywhere else.

9)-M

Enables cryptomode

Example: -M md5

Supports md5,sha1,sha224,sha256,sha384,sha512 algorithms

What does it do exactly?

Say, you have a video file I_am_not_a_pirate.mp4 in a folder. You open up a terminal in this folder and run “rarpar -d -M md5

Now, you have in your folder another folder(-d) “d5fa019be35a69dc4daf279bab53412c” which contains archives and par2 files named “d5fa019be35a69dc4daf279bab53412c” and your file in this archive is renamed to hash as well. Original file

that was RARed and PARed is kept intact

“d5fa019be35a69dc4daf279bab53412c” – is hash made from your file’s name.

The script also generates a dummy NFO files so that you know based on it’s name what exactly this hashed release is.

Hash is based only on a filename\folder name\+suffixes\prefixes (see below). Extensions are not taken into account

10)-min

Processes only files larger than X MB

Example: -min 5

11)-n

Uses old style rar naming

Example: rarpar -d -M md5 -n

Results:Β Old style rar naming convention

Old style rar naming is on the left

12)-N

Creates NFO for processed media files. Works only in a file mode (default)

When you have this option enabled your NFO file created for media files will contain mediainfo details

13) -NN

New name. Rename file/directory during processing. Shouldn’t be used with multiple items. Disabled in recursive mode.

Example: rarpar -d -M md5 -v 15 -V -NN MPIAA.2013.DVDRip -p [PORN] -s -COOLGROUP --sfv -P pass

Results:New Name

As you can see, hash is calculated from [PORN]MPIAA.2013.DVDRip-COOLGROUP The file in archives had it’s original name intact because we used hash key (-M) and a password(-P) If we didn’t use β€œ-M” then we would have rars and par2 files named as [PORN]MPIAA.2013.DVDRip-COOLGROUP Same renaming can be done to a single subdirectory. Don’t forget you should use -NN only for processing one item (file\folder)

14)-O

Sets output directory

Example: rarpar -d -M md5 -O /media/truecrypt3/UPLOAD/ -N

Results:Β Set output directory

On the left is the output directory, on the right the folder where we opened terminal and ran the script

Nfo file contains mediainfo details of this particular file

15)-p

A filename’s prefix

Just in case if you need one for whatever reasons

Example: rarpar -d -M md5 -O /media/truecrypt3/UPLOAD/ -N -p [fuckYouMpiAA]

Results:Β Prefix

Hash is calculated from “[fuckYouMpiAA]I_am_not_a_pirate

16)-P

Creates password protected archives.

Example: rarpar -d -M md5 -O /media/truecrypt3/UPLOAD/ -N -p [fuckYouMpiAA] -P youllneverknow

Same results as in #13 but passworded + this time a file\folder inside archives is not renamed to hash because RARs are password protected.

17)-r

PAR2 recovery in % (default 10)

Hint: For files\folders >=4Gb it’s better to use 5, not 10

Example: rarpar -r 5 -d -M md5 -O /media/truecrypt3/UPLOAD/ -N -p [fuckYouMpiAA] -P youllneverknow

18)-R

Recursive mode. Works only when processing files

Allows the script to process files in subfolders.

19)-s

A filename’s suffix. Pretty much the same as prefix

Example: rarpar -d -M md5 -O /media/truecrypt3/UPLOAD/ -N -p [fuckYouMpiAA] -P youllneverknow -s -CoolGROUP

Results:Β Suffix

20)--sfv

Creates SFV file

Example: rarpar -d -M md5 -O /media/truecrypt3/UPLOAD/ -N -p [fuckYouMpiAA] -P youllneverknow -s -CoolGROUP --sfv

21)--subdir

The script processes subdirectories, not files. Not used with -N

Example: rarpar -d -M md5 -O /media/truecrypt3/UPLOAD/ -p [fuckYouMpiAA] -P youllneverknow -s -CoolGROUP --sfv --subdir

Results:Subdir mode

Hash is calculated from “[fuckYouMpiAA]wtf_do_we_have_here-CoolGROUP

22)-u

Uniform par2 recovery file size.

Hint:Not a good idea. Leave it

23)-v (default 50)

RAR volume size in MB.

At your own discretion.

Example: rarpar -d -M md5 -O /media/truecrypt3/UPLOAD/ -p [fuckYouMpiAA] -P youllneverknow -s -CoolGROUP --sfv --subdir -v 15

24)-V

If turned on, you get to see detailed processing information (% and what not) in a terminal.

It is advisable to turn it on. This way you can monitor the process if needed.


There are quite a lot of posts dedicated to rarnpar in this blog, but if you want to get the most recent version, you could use following link: https://tad-do.net/wp-content/rarnpar. If you would like, for some reason, to get previous versions, you could always check a post dedicated to that particular version:

LICENSE

The script is distributed under GPL v2 license with NO WARRANTY WHATSOEVER. Feel free to modify it to suit your needs.

20 thoughts on “rarnpar”

  1. Works like a charm!

    Just a couple of comments:

    Would it be possible to adapt the PAR2 CPU throttling parameters ? my multi-threaded multi-core PAR2 pretty much kills the server while creating the PARs πŸ™‚

    -t is the option in my PAR2, -t- sets it in single core mode.

    The possibility of overriding newsmangler options would be great.

    another nice feature would be to be able to start the posting to the server while the PAR2 recovery creation is running in parallel, and post those after the main UL’s been done.

    and finally, the application not assume current work dir to be *the* dir it’s supposed to work with πŸ˜‰

    thnx for a great piece of code

    1. Thank you for the feedback.
      I am not sure if -t option is available in all par2 implementations. I’d like to use only universal options in the script. I will look into this, but in the meantime you could try using either single-core PAR2 or set the limits with ulimit.

      Newsmangler options are easily overwritten via its config file, therefore I see no need for additional parameters.

      Bash isn’t really suited for multi-threading. Yes, it’s doable, but I would probably just rewrite the script in ruby, if I wished to implement this feature.

      And what do you mean by “the application not assume current work dir to be *the* dir it’s supposed to work with”? If you don’t specify a directory, script isn’t using the current one?

  2. Hi Tad,
    can you explain to me how i process a complete folder? It is simply not working for me.

    I got a Movie incl. nfo, Sample etc. in the folder /home/user/MyMovie/
    Now i want that the folder MyMovie will be packed into one rar set.

    What is the correct commandline for that? Always whe i try it i get a error “No Files to process”.

  3. Hi,

    Nice script. How trustworthy is the RAR password protection? Do we know what algorithm it uses? How about using 7-zip instead? It works great on many platforms. It supports up to AES-256 encryption and can encrypt headers too.

    1. Hi,
      Rar is used here(and notice how the script is called πŸ™‚ ) not because it’s highly secure or offers great compression, but because it’s the current de facto standard for usenet uploads.

    1. If I click your link, I will see what file is in your archive. Cryptorenaming was designed specifically to counter that. Just specify a password and files won’t be renamed, only rars will be.

      1. Yes, the rar contents are visible. My suggestion aims to slow down dmca take downs while still making the post available to all.
        I’m looking for an obfuscation method similar to the functionality you originally had in the initial versions of rarnpar (if i remember correctly)
        imo passwords are a blight on usenet and I’ll never use them.

  4. So happy to find this but I’m lost.
    Figured how to use it but not quite the way I want.
    2 things I can’t figure out.
    I have multiple files is say /home/user/movie/MYMOVIE and I want my all the files in the subdir MYMOVIE to be in one .rar file with a _specific_ name (ie THISISMYMOVIE.rar) with appropiately named .par files. I don’t want the original filenames in the .rar filename.
    So:
    1) a single .rar containing all files in a directory
    2) A specific name for the file

    Thanks for all your work
    Glaft

    1. To have all files in one archive, you need to use –subdir switch and use /home/user/movie as your starting directory. -NN option is specifically designed for a case like yours, when you need to rename MYMOVIE to THISISMYMOVIE.

      I hope that it helps.

  5. Hi–

    I’m trying to use rarnpar from Ruby. I generate the command that I wish to execute in Ruby, but the command fails. I’m running on a Mac using latest OS X.

    When I run the same command from the shell, it executes just fine. I’ve looked at that various ways to calll/exec/fork, etc. and I keep getting same error. My sh scripting isn’t advanced enough to understand much of the line in question (the one raising the error).

    The rarnpar command I’m running is this:
    rarnpar -d -e ‘mp4’ -min 5 -N -NN “Axxxxxxxxxxx.S01E05.Title.DVDRip.x264” -s -CPM –sfv -v 15 -V

    The line being called as an error is this:
    /usr/local/bin/rarnpar: line 659: syntax error near unexpected token `<'
    /usr/local/bin/rarnpar: line 659: ` done < <(find -L ${MAXDEPTH} -type d -name "*" ! -name ".*" | sed -e 's/^\.//g' -e 's/^\///g')'

    This is the error I get in my IDE, but I get the same error running the Ruby script from the CLI.

    So: called from Ruby I get an error; run from CLI it runs fine.

    Ideas?

    thanks
    ajw

    1. Hi.
      How are you trying to execute it? I would do it like this:
      `bash -c ‘the whole rarnpar command goes here’`
      This way you will force to execute it via bash. Hopefully it will solve the problem.

      P.S. “About” page to ask such question? Interesting choice πŸ™‚

      1. Hi Tad–

        bash -c ‘command and args’ worked fine.

        Please feel free to move these comments somewhere more appropriate (or remove them).

        thanks

        ajw

  6. Hi–

    Wasn’t expecting to see my comment appear here (on your about page). Please feel free to either move it or leave it unapproved, etc.

    thanks

  7. Hi, I’ve been using your script with sanguinews for a week.
    I’m trying to alter the sanguinews.conf file but can’t locate it.
    When I installed it for the first time I got a message that it was moved to home directory (or root directory) but can’t find it anywhere.
    I found the original .conf file in the sanguinews-0.80.1 directory but changing that didn’t change anything.

    1. Hi! The file you are talking about is a hidden one. In a terminal you could open it with vi ~/.sanguinews.conf (replace “vi” with your favourite text editor).

  8. Hello,

    It’s possible to add a new fonction ?

    I want to change the extension name after the par2 creation (eg: walt.part01.rar -> walt.01 / walt.part02.rar -> walt.02 / walt.part03.rar -> walt.03 / …)
    Can you add this function, or tell me how can I add this ?

    Thanks for your great job !

Leave a Reply

Your email address will not be published. Required fields are marked *

Are you human? *

The Way of Tad