Fix broken link generation for KERNEL projects.
Using $1 in projectsmirrors sub was still referencing the caller $1
instead of the remaining args of projectsmirrors sub.
Use shift and put the second arg of projectsmirrors sub in $append to
correctly handle the sub args.
Fixes: 465cf358881c ("scripts/download.pl: detach mirror URLs from script file")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Detach URL from download script to make it easier to change and update
the URLs.
The mirror list is moved to a JSON file in the scripts directory called
projectsmirrors.json.
Link: https://github.com/openwrt/openwrt/pull/15745
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Add GNU's redirector which automatically redirect user to nearby online
mirror.
Signed-off-by: Sahil Dhiman <sahil@hopbox.in>
Link: https://github.com/openwrt/openwrt/pull/15557
Signed-off-by: Robert Marko <robimarko@gmail.com>
Remove the stale site definitions from @APACHE, KERNEL etc.
* Remove site that had dropped APACHE
* Remove KERNEL site leading to wrong directory
* Remove dead sites
* Convert ftp/http URLs to https, if possible. Remove duplicate
Signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>
Remove stale sites from @GNOME alias:
* remove 2 sites that have stale 3 years old content
* remove 2 sites that have dropped GNOME
* convert 2 sites from FTP to HTTP
Signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>
mirror2 now points to mirrors.gigenet.com.
This reverts commit 9fbf1751eced799a4320fed8095b78fbde2d5116.
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
The aria2c command tries to load config from
${XDG_CONFIG_HOME:-${HOME}/.config}/aria2/aria2.conf by default,
which may result unexpected behavior.
As a replacement, people can use environment variable ARIA2C_OPTIONS
to custom arguments passed to aria2c like curl and wget below.
Including --conf-path=/path/to/config.conf in ARIA2C_OPTIONS can
also set a custom config file path easily if needed.
Signed-off-by: Zhang Hua <zhanghuadedn@gmail.com>
Introduce a new option in the "Advanced configuration options" to
configure a custom download tool.
By declaring a string in "Use custom download tool" an user can force
what command to use to download package. With the string empty the
default tool used is curl, with wget as a fallback if not available.
download.pl supports 3 tools officially aria2c, curl and wget.
If one of the tool is used in this config, download.pl will use the
default args to make use of them.
If the provided string is different than aria2c, curl or wget, the command
is used as is and the download url will be appended at the end of such command.
While at it also tweak the tool selection logic and chose the tool only
once when the script is called and move aria2c specific variables in the
relevant section.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Generilize download tool check and skip other check if a download tool
has been found.
While at it also reintroduce c836ca84e8f641e10a8349a8f9b7432b33d6cec1
that was previously dropped with aria2c support.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Currently we use /dev/shm to place aria2c tmp file. This is not present
on macos. Use the openwrt tmp directory instead of the linux-only
/dev/shm to save compatibility with more os.
Fixes: d39123626931 ("download.pl: add aria2c support")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
With the introduction of aria2c support, curl and wget no longer try to
download the file from mirrors. Fix this regression by emptying the
remaining mirrors list only when aria2c is used.
Fixes: d39123626931 ("download.pl: add aria2c support")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Fix whitespace in mirror urls and replace for loop with join+map logic.
Fixes: d39123626931 ("download.pl: add aria2c support")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Use aria2c download tool by default on package download if available in
the system.
aria2c permits to use multiple mirrors and may improve download speed on
special context where servers are hard to reach.
Co-authored-by: Christian Marangi <ansuelsmth@gmail.com>
Signed-off-by: Bradford Zhang <zyc@zyc.name>
[ fix wrong var in the script and improve commit description ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Several users of wget for downloads (curl is not available in the
system) have reported broken download functionality:
wget --tries=5 --timeout=20 --output-document=- https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.10.142.tar.xz
http://: Invalid host name.
Thats all happening due to '' was passed as an argument, which got later
expanded to http://.
In the context of a list constructor '' is not nothing, it is an empty
string element. So fix it by using () as it will yield "nothing" and
thus not introduce an empty string element.
Fixes: #10692
Fixes: 90c6e3aedf16 ("scripts: always check certificates")
Signed-off-by: Jo-Philipp Wich <jo@mein.io> [shellwords() -> ()]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
When running build in verbose mode `make V=s` we can see a lot of
following warnings when curl is not available in the system:
Can't exec "curl": No such file or directory at scripts/download.pl line 77.
So lets fix it by redirecting of the stderr to null hole.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Remove flags from wget and curl instructing them to ignore bad server
certificates. Although other mechanisms can protect against malicious
modifications of downloads, other vectors of attack may be available
to an adversary.
TLS certificate verification can be disabled by turning oof the
"Enable TLS certificate verification during package download" option
enabled by default in the "Global build settings" in "make menuconfig"
Signed-off-by: Josh Roys <roysjosh@gmail.com>
[ add additional info on how to disable this option ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Ubuntu started to flag which as deprecated and it
seems which is not really standard and may vary
across Distro.
Drop the use of which and use the standard 'command -v'
for this simple task.
Which is still present in the prereq if some package/script
still use which.
A utility script called command_all.sh is implemented that
will just mimic the output of which -a.
Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
To avoid abuse, these mirrors will not be used until you enable them
manually.
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit a3325e5051307bb92c2c3feaf44b0980421eb98c)
Before this commit, it was assumed that mkhash is in the PATH. While
this was fine for the normal build workflow, this led to some issues if
make TOPDIR="$(pwd)" -C "$pkgdir" compile
was called manually. In most of the cases, I just saw warnings like this:
make: Entering directory '/home/.../package/gluon-status-page'
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
[...]
While these were only warnings and the package still compiled sucessfully,
I also observed that some package even fail to build because of this.
After applying this commit, the variable $(MKHASH) is introduced. This
variable points to $(STAGING_DIR_HOST)/bin/mkhash, which is always the
correct path.
Signed-off-by: Leonardo Mörlein <me@irrelefant.net>
