[NOT-TF][BitTornado][Issue] set priority + torrents with multiple files

[b4rt]

there is a issue (i would say its a bug ~) in the file-priority feature.
i did not post it into this thread although its the same reason because i wanted to be concrete in subject+text.

i think i found out how to reproduce and fix it (but that fix is no solution~).
(at least i tried it many times and i observed always the same, so i am "quite sure" about this)

following scenario:

• 1. you want to seed a multi-file-torrent (a torrent with multiple files in it, a "directory-seed" (i dont know the official term for that ~)
• 2. prepare data, load torrent-file into flux
• 3. start torrent with priority-selection enabled : dont choose all files, start torrent only with a few files selected (simply just select first file and run torrent)

tornado will seed that file you selected ... but:

• if you stop that torrent, the files you didnt select will be deleted (lets assume you selected only the first file to seed, bye bye to the rest-files in the dir at/after stop)
• you restart the torrent : no matter what files you choose in the priority-tree , you only can seed the file(s) you did choose the first time not selecting all files in the torrent (not because the params of the dir-tree are false/are passed bad to tornado.. the rest-files are simply missing.. just check webserver-logs to see errors from tornado about not finding files)

the same will happen if you delete+reinsert the .torrent-file or/and if you delete any .prio-files that  exist. (may exist.. in my tests they never showed up)
i was wondering for quite some test-runs why this happens because even if i deleted "all" by hand to ensure there are no "old settings" the error still was present.

my last try was to delete tornados cache-files ... and that "fixed" it.
the next start i had the usual "checking data..." of a fresh torrent and:

• i could seed 100% (all files) of the torrent again
• when stopping the torrent nothing was deleted

actually i dont know if it where the files in "/usr/local/torrent/.BitTornado/datacache" or the files in "/usr/local/torrent/.BitTornado/piececache" as i deleted both at once in that test-run.

first thoughts (of a flux+tornado-noob ):

• i dont know if there are any references from a particular torrent to its cache-files avalaible. (of course there must be... inside tornado) deleting cache of all torrents per hand is no good fix.
• maybe tornado wants to be told he has to delete/rebuild "cached-settings".
• maybe tornado wants to be told about not deleting stuff.

regards,
b4rt

[ryaner]

This is actually a stupid BitTornado feature / bug. Caused me so much problems in the past. It will delete files that are already downloaded if you set them to not be downloaded in the priority array. There is no way to stop this short of modifying BitTornado.

[b4rt]

This is actually a stupid BitTornado feature / bug. Caused me so much problems in the past. It will delete files that are already downloaded if you set them to not be downloaded in the priority array. There is no way to stop this short of modifying BitTornado.

its nasty, especially when seeding. (glad me had a backup ~)

what i ensured after doin some more tests is:
the files are deleted when stopping the torrent with file-priority. (they are not deleted at once after starting it)
so there may be a dirty way to workaround this problem (wo patching tornado by doing "something" to prevent deleting data).

EDIT:
http://forums.degreez.net/viewtopic.php?t=7051
http://forums.degreez.net/viewtopic.php?t=5181
http://forums.degreez.net/viewtopic.php?t=2147


regards,
b4rt

[b4rt]

prevent deleting of "do not download" files needs a little change in tornado.
i tested it with my flux (2.1 and included tornado 0.3.15) and some multi-torrents and it works although i am not sure so far if there are any (no-good) sideeffects from this.

edit /var/www/TF_BitTornado/BitTornado/BT1/FileSelector.py
find this:

Code:

    def finish(self):
        for f in xrange(self.numfiles):
            if self.priority[f] == -1:
                self.storage.delete_file(f)

and replace with:

Code:

    def finish(self):
        return

(because i dont know how the python-compiler works (how smart he is ~) i deleted existing .pyo/.pyc after changing FileSelector.py)

still a problem:
the "upgrade-resume" still doesnt work (i think because of the cache-files of the tornado)
with "upgrade-resume" i mean for example:

• dir-seed with multiple files. you start to seed only the first file
• you stop the torrent (with "patched" tornado nothing is deleted here now).
• you restart the torrent, now choosing files 1+2 to seed

-> tornado will still seed only the first file ignoring priority-settings. (stop,delete tornado-cache, restart works in that case)

regards,
b4rt

[ryaner]

Nice find on the priority thing. Only sideeffect is left over directories from non downloaded files (some users have said this is a problem in the past but I'd prefer not to lost data over having extra dirs).

The resume this seems like a BitTornado bug. It should be refreshing it's cache files on a restart and normally will when downloading data.

[b4rt]

Nice find on the priority thing. Only sideeffect is left eing stuff is deleteeover directories from non downloaded files (some users have said this is a problem in the past but I'd prefer not to lost data over having extra dirs).

The resume this seems like a BitTornado bug. It should be refreshing it's cache files on a restart and normally will when downloading data.

i prefer to not loose data when seeding
empty subdirs are not that big problem for me.. when manually deleting stuff is deleted recursive anyway.

maybe one workaround about the cache-issue would be to delete the old cache manually (via flux eg) when that scenario applies. the problem is i dont know how tornado references the cache-files from a particular torrent. (to only delete those, delete all is no good thing)
better would be a tornado-option "rebuild-cache" but i didnt find anything like that in their forums.

regards,
b4rt

[ryaner]

The cache files store the hash and the priority settings which are causing this nice bug. There is an option for expire_cache_data which you could set to 0 or -1 to clear the cache but it'd be hard enough to know when to pass it since you don't want to pass it all the time as removing the cache files stops fast resumes.

[b4rt]

one way may be to trace via flux when priority-settings have changed and do it then.
-> dont do it if prio-settings arent touched at all
-> dont do it on first time

when does the flux delete the .prio-file ? when stopping torrent-x or restarting it ?

EDIT:
found that one, its in the function setPriority in setpriority.php.

EDIT:
another (simple) way would be do add it in advanced startup-dialog

regards,
b4rt

[ryaner]

Suppose checking the timestamp on the prio file isn't a bad idea although adding it to advanced startup is simpler.

Saying that though, I'm unsure if the option fully works as I haven't tested it since I'm having some minor space issues on my server.

[b4rt]

Suppose checking the timestamp on the prio file isn't a bad idea although adding it to advanced startup is simpler.

Saying that though, I'm unsure if the option fully works as I haven't tested it since I'm having some minor space issues on my server.

i just tested "the simple way" :

• add checkbox to advanced dialog
• start tornado with --expire_cache_data 0 / --expire_cache_data -1 when checked

result: did not work for me, tornado did not seed "more". only the files(s) selected on first partial select.
(i think i saw the file in "datacache" disappeared with --expire_cache_data 0 but unsure, nevertheless tornado only seeded part of the torrent)
i think the only option is to delete the dir in "piececache" manual if needed. afaic the name of the dir is a string with the infohash of the torrent.

i dont know atm if i dig further (implementing that manual delete) as its not really my every-day use and you can workaround by manual deleting the cache.

regards,
b4rt