More fine grained speed limit for copy operation

Here you can propose new features, make suggestions etc.

Moderators: white, Hacker, petermad, Stefan2

Post Reply
MarekKnapek
Junior Member
Junior Member
Posts: 7
Joined: 2019-09-07, 06:08 UTC

More fine grained speed limit for copy operation

Post by *MarekKnapek »

Hi, I do have following use case: I want to copy (large) files via very poor connection. Maybe between my PC and USB stick or between my PC and network NAS or between my PC and other PC maybe via RDP via VPN via 4G mobile data. During copy operation the other device is barely usable because all bandwidth is consumed by the copy operation. To mitigate this I'm using background transfer manager (or whatever it is called) by pressing F5 followed by F2 and limit the transfer speed there. Total Commander implements the rate limiting in following way: read 1MB, write 1MB, wait for some time, and then repeat: read 1MB, write 1MB, wait, read 1MB, write 1MB, wait. (I used sysinternals' procmon tool to discover the read/write amount.)

The problem: 1MB is too coarse. I cannot do anything during the operation (file write in my case) and experience using the other device via TC (or File Explorer) is very poor. In fact, I'm writing this post from computer I'm controlling via the RDP+VPN+4G setup and oh boi, it is unpleasant, the screen freezes for seconds at a time, I cannot see what I'm writing.

The suggestion: In case user selects low transfer speed limit, make the read + write chunks much smaller than the 1MB it is currently. Maybe if the speed is smaller than 1MB/s use 64kB chunks? Some tuning required. Perhaps add this as user configurable option into .ini files, no need to expose it in GUI.

Best regards, Marek.
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: More fine grained speed limit for copy operation

Post by *Usher »

1. Go to Configuration options and select proper copy method.
2. Additionally you can use Background Transfer Manager and limit transfer bandwith (maximal copy speed)
Andrzej P. Wozniak
Polish subforum moderator
MarekKnapek
Junior Member
Junior Member
Posts: 7
Joined: 2019-09-07, 06:08 UTC

Re: More fine grained speed limit for copy operation

Post by *MarekKnapek »

Usher wrote: 2022-09-25, 15:03 UTC 2. Additionally you can use Background Transfer Manager and limit transfer bandwith (maximal copy speed)
Yes, that is exactly what I'm doing already, I'm limiting the copy to be 300kB/s, but the device is not responsive, because TC copies in 1MB chunks and inserts pause between the chunks to achieve the desired speed limit. It takes about 3s to write 1MB chunk, so the device is unusable for 3s then all the other commands go thru and then again 3s freeze.
Usher wrote: 2022-09-25, 15:03 UTC 1. Go to Configuration options and select proper copy method.
Yes, this helps, thank you. But I'm worried about other fast disk-to-disk transfers to be negatively affected. So I need to find single value to work well with both fast and slow connections. I was expecting there would be special settings for the rate-limiting copy operation. On local computer I would like the copy to be fast and efficient as possible, and the rate-limited copy operation to be more fine-grained configurable separately.

So thank you for pointing me to the copy method settings, I can live with single setting for both fast and crappy connections. BTW speaking about efficiency, is there reason why TC doesn't use memory mapped files (instead of ReadFile/WriteFile)?

Thank you, Marek.
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: More fine grained speed limit for copy operation

Post by *Usher »

Just try with two following settings:
1. For slow copy - use compatiblity mode with drive letters provided and USB checked on. In addition use BTM (background transfer) with speed limit always turned on by adding SpeedLimitEnabled=1 to [Configuration] section in wincmd.ini.
2. For fast copy - use standard copy mode with big file copy mode selected - it should use memory mapping.

There are many topics about copy settings so you can search and read if you still need more info.
Andrzej P. Wozniak
Polish subforum moderator
Post Reply