[9.20b1 & older] Background transfer manager limiter bug

The behaviour described in the bug report is either by design, or would be far too complex/time-consuming to be changed

Moderators: white, Hacker, petermad, Stefan2

vasevase
Junior Member
Junior Member
Posts: 6
Joined: 2018-05-04, 22:20 UTC

[9.20b1 & older] Background transfer manager limiter bug

Post by *vasevase »

I want to limit (non-network) copying speed operation, but seems BTM limiter's checkboxed value is ignored. The same job works fine with alternative file manager.
User avatar
sqa_wizard
Power Member
Power Member
Posts: 3854
Joined: 2003-02-06, 11:41 UTC
Location: Germany

Post by *sqa_wizard »

You are right concerning TC 9.12, but are you sure about TC 9.20b1 ?
history.txt wrote:03.05.18 Release Total Commander 9.20 beta 1 (32/64)
...
26.03.18 Fixed: Copying files, background transfer manager: Speed limit didn't work any more for the second file and below (32/64)
#5767 Personal license
vasevase
Junior Member
Junior Member
Posts: 6
Joined: 2018-05-04, 22:20 UTC

Post by *vasevase »

sqa_wizard wrote:are you sure about TC 9.20b1 ?
Yes. It took me about 10 seconds to copy two files from/to my old HDD. The limiter's value was set to default 5kb/s. Expected time ≈ 30,000kb/5 = 6,000 seconds. I'm using NT5.x x86 OS if it matters. Though I got the same result on virtual Windows 7 x86.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

Unfortunately it's not possible to set a speed limit which is so different from the maximum possible speed. Try something like 10000kB/s when the maximum is 30000kB/s.

Why? TC calls CopyFileEx for copying, and then slows down the copying by calling sleep in the callback function. TC has no influence on the buffer size used in CopyFileEx, so when the callback function gets called by Windows, a big chunk of the file has already been copied.
Author of Total Commander
https://www.ghisler.com
User avatar
DrShark
Power Member
Power Member
Posts: 1872
Joined: 2006-11-03, 22:26 UTC
Location: Kyiv, 68/262
Contact:

Re: [9.20b1 & older] Background transfer manager limiter

Post by *DrShark »

vasevase wrote:The same job works fine with alternative file manager.
Without name of software where "it works" and the copy settings you use there, it's almost impossible to help here.

Only thing I can suggest is to try Total Commander option "Use default method only" (Configuration -> Copy/Delete) - maybe then speed limit will work better for you . This copy method doesn't use CopyFileEx:
Help wrote:The default built in copy method (via ReadFile/WriteFile) is relatively slow, but also reliable. A larger block size may speed up the copying. When copying within the same drive, a larger buffer size should be chosen than when copying between two drives, so the read/write head doesn't have to jump between source and target all the time.

2ghisler(Author)
Maybe for CopyFileEx method, you can add dynamic recommended min speed limit shown in BTM window?
Donate for Ukraine to help stop Russian invasion!
Ukraine's National Bank special bank account:
UA843000010000000047330992708
vasevase
Junior Member
Junior Member
Posts: 6
Joined: 2018-05-04, 22:20 UTC

Post by *vasevase »

DrShark wrote:
vasevase wrote:The same job works fine with alternative file manager.
Without name of software where "it works" and the copy settings you use there, it's almost impossible to help here.
Are you the developer, no? Maybe I should also post the source code here or make a patch, for free? Come on! :roll:
ghisler(Author) wrote:Unfortunately it's not possible to set a speed limit
But your competitors did it well. I hope you'll find a solution. Thanks for the answer and for your porgram too.
User avatar
DrShark
Power Member
Power Member
Posts: 1872
Joined: 2006-11-03, 22:26 UTC
Location: Kyiv, 68/262
Contact:

Post by *DrShark »

vasevase wrote:Are you the developer, no?
This is both a user-to-developer and user-to-user support forum, if you're interested only in help from TC developer, then it's better to use support e-mail (see Total Commander menu Help->About Total Commander).
Donate for Ukraine to help stop Russian invasion!
Ukraine's National Bank special bank account:
UA843000010000000047330992708
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

2vasevase
Sorry, seems like TC is not meant for you. I don't plan to support meaningless functions, like copying locally with 4 kBytes/s. The speed limiter is meant to copy files in a way that they don't take 100% of the bandwidth, e.g. in a local network or from/to an ftp server. Here it makes sense to reduce the speed to e.g. 1/2 or 1/4 of the full speed. But it doesn't make sense to reduce the speed to 1/1000000 of the full speed. If you really need this, please look elsewhere.
Author of Total Commander
https://www.ghisler.com
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

So, can anyone test that the original bug has been fixed, with _realistic_ values? Thanks.
Author of Total Commander
https://www.ghisler.com
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

BUMP!
Author of Total Commander
https://www.ghisler.com
vasevase
Junior Member
Junior Member
Posts: 6
Joined: 2018-05-04, 22:20 UTC

Post by *vasevase »

So, can anyone test that the original bug has been fixed, with _realistic_ values? Thanks.
It not works correct with
SpeedLimitEnabled=1
SpeedLimit=5.0

right after start of downloading (TCx86 edition b3).
It works correct ONLY when you pause the STARTED transfer, uncheck the limiter's checkbox,
check it back and then resume the task (tested with network speed analyzer).
So I guess you need to remove this "re-start" step necessity (for local operations too).
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

Unfortunately I cannot reproduce your problem. I have tried with a more reasonable speed limit since I don't use a modem any more to access the internet (e.g. SpeedLimit=1000), and it works fine.

Can anyone else test it please?
Author of Total Commander
https://www.ghisler.com
User avatar
Dalai
Power Member
Power Member
Posts: 9364
Joined: 2005-01-28, 22:17 UTC
Location: Meiningen (Südthüringen)

Post by *Dalai »

I don't see any point in testing really low speed limits. I use a speed limit of 10,000 (10 MB/s) all the time to transfer files from/to my server within the same LAN. Sometimes I use lower limits like 5 MB/s or something. However, I use copy mode for large files, which might also be a factor. What's also important to note is that the OS might have the transferred file already in its cache, so the limit can't be set right at the beginning of the transfer, which could make it look like the limit isn't active.

But I'm wondering if TC is accounting for internet connection speeds, in particular when dealing with FTP since the internet connection can have a very different speed limit than the local connection (GBit in case of LAN).

Regards
Dalai
#101164 Personal licence
Ryzen 5 2600, 16 GiB RAM, ASUS Prime X370-A, Win7 x64

Plugins: Services2, Startups, CertificateInfo, SignatureInfo, LineBreakInfo - Download-Mirror
vasevase
Junior Member
Junior Member
Posts: 6
Joined: 2018-05-04, 22:20 UTC

Post by *vasevase »

ghisler(Author) wrote:Unfortunately I cannot reproduce your problem.
Try these steps:
- connect to some FTP with large enough file
- select this file with ^ v keys (don't press "ins")
- click "commands", "background transfer manager"

the window will appear
- now press "+" > "add selected items.." > set your path and press "OK"

The transfer speed will be at max even if the limiter has previously been set to "ON".
But it'll slow down after the manipulations I've listed in my previous post.

And yeah, it works correct (at limited speed w/o any manipulations) if you push "F5" and check the "download in background..." checkbox.
Dalai wrote:OS might have the transferred file already in its cache
Impossible due to my low internet speed connection.
Dalai wrote:I don't see any point in testing really low speed limits.
Cuz there're some people, who live outside of Switzerland (surprise!). :lol:
As for "realistic" things - that value was set by default. So it's just for test purpose, not for regular use.
User avatar
Dalai
Power Member
Power Member
Posts: 9364
Joined: 2005-01-28, 22:17 UTC
Location: Meiningen (Südthüringen)

Post by *Dalai »

You see, that's why it's so important to describe the steps you have to take to reproduce a bug as exactly as possible. All the time you were talking about the background transfer manager called before initiating the transfer, not the one called by the normal copy procedure.

With the steps you posted now I can confirm the issue, even with the less sensible speed limits, at least for FTP transfers (local ones work fine).

Regards
Dalai
#101164 Personal licence
Ryzen 5 2600, 16 GiB RAM, ASUS Prime X370-A, Win7 x64

Plugins: Services2, Startups, CertificateInfo, SignatureInfo, LineBreakInfo - Download-Mirror
Post Reply