Total Commander and wget, curl, yt-dlp: downloading files/video from the Internet

English support forum

Moderators: Hacker, petermad, Stefan2, white

Post Reply
User avatar
beb
Power Member
Power Member
Posts: 580
Joined: 2009-09-20, 08:03 UTC
Location: Odesa, Ukraine

Total Commander and wget, curl, yt-dlp: downloading files/video from the Internet

Post by *beb »

[1] Wget: download URLs found in local FILE (%N a file under cursor in Total Commander) into the active panel location
User command (in usercmd.ini)

Code: Select all

[em_wget_tc]
cmd=%commander_path%\Plugins\app\Downloads\wget.exe
param=-S -N --no-if-modified-since --no-hsts --user-agent="Mozilla" -i %N
[2] Wget: download URL [copied] in the clipboard into the active panel location
User command (in usercmd.ini)

Code: Select all

[em_wget_clip]
cmd=cmd /c for /f "tokens=* delims=" %i in ('powershell -command "Get-Clipboard"') do %commander_path%\Plugins\app\Downloads\wget.exe -S -N --no-if-modified-since --no-hsts --user-agent="Mozilla" "%i"
Notes:
wget parameters being used in the examples are pretty common and suitable for most user cases without the need of modification.
E.g., if a referer is required, the latter user command could be put as follows:

Code: Select all

[em_wget_clip_referer]
cmd=cmd /c for /f "tokens=* delims=" %i in ('powershell -command "Get-Clipboard"') do %commander_path%\Plugins\app\Downloads\wget.exe -S -N --no-if-modified-since --no-hsts --user-agent="Mozilla" --referer "%i" "%i"
Environment:
Windows 10 64-bit,
Total Commander 10.52 64-bit,
Wget 1.21.3 64-bit (from here: https://eternallybored.org/misc/wget/releases/)

NB Electricity in my city has gone so having pretty limited time until complete blackout I'm just leaving this post unfinished for a while.
#278521 User License
Total Commander [always the latest version, including betas] x86/x64 on Win10 x64/Android 10/15
User avatar
beb
Power Member
Power Member
Posts: 580
Joined: 2009-09-20, 08:03 UTC
Location: Odesa, Ukraine

Re: Total Commander and wget, curl, yt-dlp: downloading files/video from the Internet

Post by *beb »

[3] yt-dlp: download [youtube, etc.] video(/audio) via [a user copied] URL in the clipboard into the %Downloads% location

User command (in usercmd.ini):

Code: Select all

[em_yt-dlp_clip]
cmd=%commander_path%\Plugins\app\Downloads\yt-dlp_clip.cmd
yt-dlp_clip.cmd:

Code: Select all

@echo off
chcp 65001
REM runtime variables parsing
Setlocal EnableDelayedExpansion
REM path;cli;app
path %path%;%commander_path%\Plugins\app\Downloads;%OneDrive%\Media\FFmpeg\bin\ytdlp

:: Downloading Video from Internet into %downloads% via URL in the Clipboard
:: Notes:
:: ffmpeg and ffprobe required for merging separate video and audio files
::                    as well as for various post-processing tasks
:: for correct function put yt-dlp and ffmpeg executables next to .cmd or in %path%
:: Downloads:
:: yt-dlp Builds -- https://github.com/yt-dlp/yt-dlp/releases
:: ffmpeg Builds -- https://github.com/yt-dlp/FFmpeg-Builds/releases

:: Locate %downloads% folder
for /f "tokens=2,*" %%c in (
'reg query "HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders" /v {374DE290-123F-4565-9164-39C4925E467B}'
) do set downloads=%%d

:: Get url from the clipboard
for /f "delims=" %%i in ('powershell -command "Get-Clipboard"') do set url=%%i

yt-dlp.exe -F -S vext "%url%"
if ERRORLEVEL 1 (goto ERROR) else (goto OK)

:ERROR
echo.
:: if there's no url wait 7 seconds then exit
@echo there's no url in the clipboard, exiting
ping -n 8 ::1 >nul
exit/b

:OK
echo.
set "num="
set /p num="Enter videoID+audioID (for example 137+140) or leave empty for bestvideo+bestaudio: "
echo.
if not defined num (
:: if there's no explicit id - download the best quality
yt-dlp.exe -f "bestvideo[ext=mp4]+bestaudio[ext=m4a]/best" "%url%" -P %downloads% -o "%%(title).50s.%%(ext)s" --no-part
if ERRORLEVEL 1 (goto OK)
) else (
yt-dlp.exe -f %num% "%url%" -P %downloads%  -o "%%(title).50s.%%(ext)s")
if ERRORLEVEL 1 (goto OK)
echo.
echo Downloaded in %downloads%
color 27
:: explorer %downloads%
echo.
:: wait 3 seconds then exit
@echo wait a bit
ping -n 4 ::1 >nul
exit/b

Edit:
Credits: yt-dlp_clip.cmd is based on the
gist.github.com/wincmd64/yt-dlp_v2022.bat
code.
#278521 User License
Total Commander [always the latest version, including betas] x86/x64 on Win10 x64/Android 10/15
User avatar
beb
Power Member
Power Member
Posts: 580
Joined: 2009-09-20, 08:03 UTC
Location: Odesa, Ukraine

Re: Total Commander and wget, curl, yt-dlp: downloading files/video from the Internet

Post by *beb »

Wget: fully PowerShell-powered command to download an URL [that has been copied by a user and resides] in the clipboard into the active panel location.
usercmd.ini:

Code: Select all

[em_wget_clip]
cmd=Powershell -ExecutionPolicy Bypass; set bin -value "$env:commander_path\Plugins\app\Get\wget.exe";set url -value (Get-Clipboard);
param=start $bin """-S -N --no-if-modified-since --no-hsts -U=Mozilla --referer="$url" "$url""" -NoNewWindow -Wait #;pause
NB uncomment "#;pause" part to see what's going on for a bit longer if needed.
NBNB the command covers most of user cases to simply download a file from the Internet keeping the server timestamp (if provided by the server)
Last edited by beb on 2023-04-16, 17:04 UTC, edited 2 times in total.
#278521 User License
Total Commander [always the latest version, including betas] x86/x64 on Win10 x64/Android 10/15
Jon Canale
Member
Member
Posts: 141
Joined: 2003-02-07, 12:54 UTC

Re: Total Commander and wget, curl, yt-dlp: downloading files/video from the Internet

Post by *Jon Canale »

I get a ~700k file with no extension. File seems to be html code, not video. Tried a couple of different Youtube files. From Share button I copied the video link and I also tried the "Copy video URL at the current time" by right-clicking on the video. Using the last Powershell command version.

Am I doing something wrong?
User avatar
Gral
Power Member
Power Member
Posts: 1600
Joined: 2005-01-26, 15:12 UTC

Re: Total Commander and wget, curl, yt-dlp: downloading files/video from the Internet

Post by *Gral »

Just guessing - you used wget to download files from youtube while you should use yt-dlp - you didn't specify it.

I use the above-mentioned programs in a different, IMO much more comfortable way - through internal associations

Code: Select all

Filter1=*.wg
Filter1_open=wget.exe -U "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:112.0) Gecko/20100101 Firefox/112.0" -i "%1" -t 1 --no-check-certificate

Filter2=*.yt
Filter2_open=yt-dlp.exe --no-check-certificate -a "%1" -i --console-title
After that you will be able to start the download simply by pressing Enter on the file with links - *.wg file for wget and *.yt file for Youtube and other video sites.
PowerShell not needed.

Note: this is an extremely simplified example, you can learn how switch for these programs works and tune for your needs.
Jon Canale
Member
Member
Posts: 141
Joined: 2003-02-07, 12:54 UTC

Re: Total Commander and wget, curl, yt-dlp: downloading files/video from the Internet

Post by *Jon Canale »

Yes, I used wget to try to download Youtube files. I'll look for yt-dlp and try that approach. Thanks.
Jon Canale
Member
Member
Posts: 141
Joined: 2003-02-07, 12:54 UTC

Re: Total Commander and wget, curl, yt-dlp: downloading files/video from the Internet

Post by *Jon Canale »

I could dl a youtube short using "[em_yt-dlp_clip]" method, but not a long video (1/2 hr duration). I get this message:

ERROR: unable to download video data: [WinError 31] A device attached to the system is not functioning
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
PermissionError: [WinError 31] A device attached to the system is not functioning

Seems to be something to do with code page, but fix is supposed to be 'chcp 65001' in the cmd file, but it is already there??
User avatar
solid
Power Member
Power Member
Posts: 749
Joined: 2004-08-09, 11:20 UTC

Re: Total Commander and wget, curl, yt-dlp: downloading files/video from the Internet

Post by *solid »

While it may be slightly off-topic since it's not related to TC and portable, I've found this to be the easiest way to download videos from YouTube or almost any other site.
This method is specifically for the Vivaldi browser, but it can also be used in any other browser, although not integrated as it is in Vivaldi.

Integrating Youtube downloader into Vivaldi (tutorial)
Jon Canale
Member
Member
Posts: 141
Joined: 2003-02-07, 12:54 UTC

Re: Total Commander and wget, curl, yt-dlp: downloading files/video from the Internet

Post by *Jon Canale »

I was able to download a full video from a different youtube channel, so I'm wondering if the WinError 31 is from a channel who protects their video's. I'm not an expert on youtube, but I assume there is some ability to prevent downloads if the owner wishes it - I think that's what I ran into.

Anyway, thanks to the responders, Gral and solid, and thanks to beb for posting this code.
User avatar
beb
Power Member
Power Member
Posts: 580
Joined: 2009-09-20, 08:03 UTC
Location: Odesa, Ukraine

Re: Total Commander and wget, curl, yt-dlp: downloading files/video from the Internet

Post by *beb »

yt-dlp: fully PowerShell-powered command to download a VIDEO [such as YouTube, etc] by an URL [that has been copied by a user and resides] in the clipboard into the Downloads location.
usercmd.ini:

Code: Select all

[em_yt-dlp_clip]
cmd=Powershell -ExecutionPolicy Bypass; si -path env:path -value $env:path';'$env:commander_path\plugins\app\Get';'$env:OneDrive\Media\FFmpeg\bin\ytdlp;
param=start yt-dlp.exe """$(Get-Clipboard) -P $((New-Object -ComObject Shell.Application).NameSpace('shell:Downloads').Self.Path) -i""" -NoNewWindow -Wait #;pause
Notes:
yt-dlp requires ffmpeg and ffprobe for merging separate video and audio files as well as for various post-processing tasks
Downloads:
yt-dlp Builds -- https://github.com/yt-dlp/yt-dlp/releases
ffmpeg Builds -- https://github.com/yt-dlp/FFmpeg-Builds/releases
Documentation:
https://github.com/yt-dlp/yt-dlp#readme
Explanations:
- uncomment [#;pause] part to see what's going on for a bit longer if needed.
- the part [si -path env:path -value $env:path';'$env:commander_path\plugins\app\Get';'$env:OneDrive\Media\FFmpeg\bin\ytdlp] adds paths to yt-dlp and ffmpeg binaries to the %PATH% environment variable for the command session; where:
[si] - alias of the Set-Item cmdlet.
[$env:path] - the %PATH% environment variable.
[';'] - the required separator between %PATH% environment variable elements.
[';'$env:commander_path\plugins\app\Get] - string adds path to the yt-dlp binary to the %PATH% environment variable.
[';'$env:OneDrive\Media\FFmpeg\bin\ytdlp] - string adds path to the ffmpeg binaries to the %PATH% environment variable.
If you keep both binaries in one place, e.g., in [';'$env:commander_path\plugins\app\Get], you can remove the part that is no longer needed, e.g., putting si part as follows [si -path env:path -value $env:path';'$env:commander_path\plugins\app\Get], respecting the [';'] %PATH% separators.
In my example paths to the binaries contain %commander_path% and %OneDrive% environment variables.
An example of a literal path might be something like this: [';'d:\something\ytdlp\bin].
In case there're spaces in the paths, [' '] would help, alike [';'d:\something' 'with' 'spaces\ytdlp\bin].
Please, consider that syntax when you would type there your own paths.

So, you have two options to involve ffmpeg in work based on the use of the %PATH% variable approach:
- keep ffmpeg binaries [ffmpeg.exe, ffplay.exe, ffprobe.exe] together with yt-dlp [yt-dlp.exe] and add the common binary path to the %PATH%.
- keep ffmpeg binaries and yt-dlp separately and add both yt-dlp and ffmpeg binaries paths to the %PATH%.
There's also the [--ffmpeg-location] parameter for the yt-dlp.exe (that allows to use ffmpeg beyond the above approach, but I won't detail it here) .
To not involve ffmpeg in work don't do the above: don't put ffmpeg binaries together with yt-dlp.exe; don't add separate ffmpeg path to the %PATH% environment variable; don't use [--ffmpeg-location] parameter.

yt-dlp.exe with ffmpeg vs yt-dlp.exe without ffmpeg, whats the difference.
My simple test results of downloading 2:23 music video from YouTube:
- some_test_music_video.mp4, file size [ 12.7MB] # without involving ffmpeg;
- some_test_music_video.webm, file size [296.7MB] # with ffmpeg plugged in.

Edit
Another example of yt-dlp.exe with ffmpeg vs yt-dlp.exe without ffmpeg
Trying to download a video from reddit (e.g. ), I'm getting:
- an mp4 video file with sound [of 12.6MB] # with ffmpeg plugged in
- an mp4 video file without sound plus a separate m4a file with sound [of the same summary size] # without involving ffmpeg
Last edited by beb on 2023-04-20, 21:27 UTC, edited 1 time in total.
#278521 User License
Total Commander [always the latest version, including betas] x86/x64 on Win10 x64/Android 10/15
KozakMak
Senior Member
Senior Member
Posts: 489
Joined: 2021-05-24, 12:39 UTC
Location: UA

Re: Total Commander and wget, curl, yt-dlp: downloading files/video from the Internet

Post by *KozakMak »

beb wrote: 2023-02-21, 18:41 UTC yt-dlp_clip.cmd:
seems like you take my script and littlу bit modified it :?
OS: Win10 | TC: latest x64
User avatar
beb
Power Member
Power Member
Posts: 580
Joined: 2009-09-20, 08:03 UTC
Location: Odesa, Ukraine

Re: Total Commander and wget, curl, yt-dlp: downloading files/video from the Internet

Post by *beb »

KozakMak wrote: 2023-04-18, 09:37 UTC
beb wrote: 2023-02-21, 18:41 UTC yt-dlp_clip.cmd:
seems like you take my script and littlу bit modified it :?
Yes. The idea of the "yt-dlp_clip.cmd" is based on your (as I now know) code [and the parts related to the yt-dlp.exe commands practically go without any modifications] that is living there:
https://gist.github.com/wincmd64 : https://gist.github.com/wincmd64/4a5463f0ba9926d38e7484b611240ccd
It's been my first experience to try yt-dlp and giving it a go.
So thank you very much.
#278521 User License
Total Commander [always the latest version, including betas] x86/x64 on Win10 x64/Android 10/15
Post Reply