1315 Commits

Author SHA1 Message Date
Álvaro Fernández Rojas
0cdc257b8d scripts: sercomm-pid: add bmips support
Apparently, Sercomm sets 2 padding bytes instead of 1 (ramips).
The HW version is a bit different than the one used for ramips.

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2023-04-09 10:01:03 +02:00
Álvaro Fernández Rojas
8382c5662e scripts: sercomm-payload: add PID file support
Allow passing Sercomm PID from file.
Until now, Sercomm PID could only be passed as an array of hex bytes.

Signed-off-by: Álvaro Fernández Rojas <noltari@gmail.com>
2023-04-09 09:55:57 +02:00
Tianling Shen
3ddf2d6612
download.pl: update cdn domain
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2023-03-15 17:44:32 +08:00
Tianling Shen
4580230440
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2023-02-06 03:37:49 +08:00
Andre Heider
19988b66d0 scripts: size_compare: print a grand total
Usefull to check the impact of treewide changes:
Change 	Local	Remote 	Package
+281	6191	5910	ubus
-547	56166	56713	procd
-13294	91544	104838	ubi-utils
~~~~~~~	total change	-13560

Signed-off-by: Andre Heider <a.heider@gmail.com>
2023-02-03 21:22:49 +01:00
Andre Heider
21242451c2 scripts: size_compare: fix support for CONFIG_BINARY_FOLDER
There is no CONFIG_BINARY_DIR, it's CONFIG_BINARY_FOLDER.

While at it, don't parse the shell compatible .config, eval it.

Signed-off-by: Andre Heider <a.heider@gmail.com>
2023-02-03 21:22:49 +01:00
Andre Heider
f56d5932d2 scripts: qemustart: support CONFIG_BINARY_FOLDER
If CONFIG_BINARY_FOLDER is set in .config, use that instead of "bin" as
the bindir.

That allows to set that config and easily run e.g.
`./scripts/qemustart armvirt 32`.

Signed-off-by: Andre Heider <a.heider@gmail.com>
2023-02-03 21:22:49 +01:00
Tianling Shen
6b1e639bfc
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2023-01-25 19:31:00 +08:00
Wenli Looi
f0e4595188 build: add ALT3 and ALT4 vendor/model/variant
This is needed for the Netgear EX7300 series v2.

Signed-off-by: Wenli Looi <wlooi@ucalgary.ca>
2023-01-25 00:42:52 +01:00
Christian Marangi
26bb4b409d
scripts: ext-tools: add option to only refresh timestamps
It's possible to have prebuilt tools already extracted. Add option to
just refresh the timestamps.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-23 19:18:05 +01:00
Christian Marangi
1506f8c322
scripts: ext-tools: follow links for host tools path
Host tools path may be a symbolic link. Use -H with find to follow path
links passed from command line to find command.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-23 19:18:05 +01:00
Tianling Shen
f3c1b22c24
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2023-01-14 13:49:28 +08:00
Christian Marangi
5f1758ef14
scripts/dl_github_archieve.py: fix generating unreproducible tar
Allign dl_github_archieve.py to 8252511dc0b5a71e9e64b96f233a27ad73e28b7f
change. On supported system the sigid bit is applied to files and tar
archieve that on tar creation. This cause unreproducible tar for these
system and these bit should be dropped to produce reproducible tar.

Add the missing option following the command options used in other
scripts.

Fixes: 75ab064d2b38 ("build: download code from github using archive API")
Suggested-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Tested-by: Robert Marko <robimarko@gmail.com>
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-12 14:59:07 +01:00
Tianling Shen
8190433295
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2023-01-07 16:43:13 +08:00
David Bauer
9ac377d0e0 scripts: add Apache fastly mirror
Add the Apache fastly mirror as preferred download source.
This service is using a dual-stacked CDN.

Signed-off-by: David Bauer <mail@david-bauer.net>
2023-01-07 01:32:58 +01:00
Tianling Shen
9b3db09fe3
download.pl: fix typo error
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-12-11 22:37:35 +08:00
Tianling Shen
3d37d7b1c0
scripts/download.pl: update mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-12-11 13:44:25 +08:00
Tianling Shen
4fb0bd80d5
Revert "download.pl: remove broken mirror"
mirror2 now points to mirrors.gigenet.com.

This reverts commit 9fbf1751eced799a4320fed8095b78fbde2d5116.

Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-12-11 12:51:49 +08:00
Tianling Shen
0eab8c68d7
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-11-27 22:05:41 +08:00
Shiji Yang
9fa8fff0af scripts: remove redundant character '0x0a' from Linksys image signature
The redundant character '0x0a' after the 192 bytes '0x00' padding broke
the factory image. We need to remove it to make things work again.

Fixes: e6769d11f3 scripts: fix missing character '0' issue in linksys image
Tested-by: Tony Butler <spudz76@gmail.com>
Signed-off-by: Shiji Yang <yangshiji66@qq.com>
2022-11-27 13:18:29 +01:00
Tianling Shen
032de6ef33
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-11-18 00:40:03 +08:00
Andre Heider
9a2d362bf7
scripts: fix dl_cleanup.py argument handling
The -w|--whitelist and -D|--download-dir arguments pass an additional value,
properly evaluate that.
Also allow to pass the download directory without -D|--download-dir, just as
the usage describes.
Finally fix spitting out the wrong error messages about those args.

Signed-off-by: Andre Heider <a.heider@gmail.com>
2022-11-15 14:45:50 +01:00
Tianling Shen
a564e8b07a
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-11-13 13:57:34 +08:00
Felix Fietkau
b8bf27e4cb scripts/mkits.sh: fix portability issue
BSD wc can output more whitespaces, which breaks the cut usage.
Replace the cut invocation with awk, which is more portable.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2022-11-09 20:43:41 +01:00
David Bauer
a208f0a9be mkits: support definition of DTB loadaddr
Support defining a per-device loadaddress for the DTB. This is required
for devices which to not align the DTB from the bootloader correctly.

Signed-off-by: David Bauer <mail@david-bauer.net>
2022-11-08 12:16:36 +01:00
Tianling Shen
fbdab6778c
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-10-31 12:45:15 +08:00
Shiji Yang
e6769d11f3 scripts: fix missing character '0' issue in linksys image
In the stock firmware of Linksys, there is a '0' after the crc checksum.
Validated on EA6350V3, EA7300 and EA7300V2's stock images.

Fixes: 892d741259 build: add a script for generating Linksys factory images
Signed-off-by: Shiji Yang <yangshiji66@qq.com>
2022-10-30 23:14:45 +01:00
Tianling Shen
d6b6dbf9e1
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-10-21 15:59:03 +08:00
Zhang Hua
a53f29b583
scripts/download.pl: pass aria2 config in ENV only
The aria2c command tries to load config from
${XDG_CONFIG_HOME:-${HOME}/.config}/aria2/aria2.conf by default,
which may result unexpected behavior.

As a replacement, people can use environment variable ARIA2C_OPTIONS
to custom arguments passed to aria2c like curl and wget below.
Including --conf-path=/path/to/config.conf in ARIA2C_OPTIONS can
also set a custom config file path easily if needed.

Signed-off-by: Zhang Hua <zhanghuadedn@gmail.com>
2022-10-20 20:38:31 +02:00
Christian Marangi
f17608ddca
scripts/download.pl: make the download tool configurable
Introduce a new option in the "Advanced configuration options" to
configure a custom download tool.

By declaring a string in "Use custom download tool" an user can force
what command to use to download package. With the string empty the
default tool used is curl, with wget as a fallback if not available.

download.pl supports 3 tools officially aria2c, curl and wget.
If one of the tool is used in this config, download.pl will use the
default args to make use of them.

If the provided string is different than aria2c, curl or wget, the command
is used as is and the download url will be appended at the end of such command.

While at it also tweak the tool selection logic and chose the tool only
once when the script is called and move aria2c specific variables in the
relevant section.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-10-20 00:35:19 +02:00
Tianling Shen
9d6e3e6c02
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-10-08 14:46:37 +08:00
Christian Marangi
5428bdc2df
scripts/ext-tools: introduce new script to install prebuilt tools
Add a simple script to make it easier to install a prebuilt tools tar.
Currently it will be used by our tools container and kernel workflow on
github.

Simple script that take a tar that contains prebuilt host tools, extract
them and refresh the timestamps to skip recompilation of such host
tools.

By default it refresh timestamps of build_dir/host and
staging_dir/host/stamp.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-09-30 22:42:18 +02:00
Bradford Zhang
e937663025
scripts/download.pl: add tsinghua and ustc mirrors
Add https://mirrors.tuna.tsinghua.edu.cn/ and https://mirrors.ustc.edu.cn/ mirrors into download.pl to speed up download in China.

Signed-off-by: Bradford Zhang <zyc@zyc.name>
2022-09-30 14:57:19 +02:00
Christian Marangi
261925a869
scripts/download.pl: generilize and simplify download tool check
Generilize download tool check and skip other check if a download tool
has been found.
While at it also reintroduce c836ca84e8f641e10a8349a8f9b7432b33d6cec1
that was previously dropped with aria2c support.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-09-28 16:55:10 +02:00
Tianling Shen
81f9f50fa4
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-09-27 17:06:04 +08:00
Christian Marangi
f1b3958d02
scripts/download.pl: fix support for aria2c download tool on macos
Currently we use /dev/shm to place aria2c tmp file. This is not present
on macos. Use the openwrt tmp directory instead of the linux-only
/dev/shm to save compatibility with more os.

Fixes: d39123626931 ("download.pl: add aria2c support")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-09-26 20:50:10 +02:00
Christian Marangi
5df60f5c24
scripts/download.pl: fix mirrors regression for curl and wget
With the introduction of aria2c support, curl and wget no longer try to
download the file from mirrors. Fix this regression by emptying the
remaining mirrors list only when aria2c is used.

Fixes: d39123626931 ("download.pl: add aria2c support")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-09-26 20:50:09 +02:00
Christian Marangi
295e0ed7a9
scripts/download.pl: fix whitespace in mirror urls and drop for
Fix whitespace in mirror urls and replace for loop with join+map logic.

Fixes: d39123626931 ("download.pl: add aria2c support")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-09-25 15:54:12 +02:00
Bradford Zhang
d391236269
download.pl: add aria2c support
Use aria2c download tool by default on package download if available in
the system.
aria2c permits to use multiple mirrors and may improve download speed on
special context where servers are hard to reach.

Co-authored-by: Christian Marangi <ansuelsmth@gmail.com>
Signed-off-by: Bradford Zhang <zyc@zyc.name>
[ fix wrong var in the script and improve commit description ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-09-25 14:56:45 +02:00
Tianling Shen
0e0795a497
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-09-23 16:56:39 +08:00
Michael Pratt
da4609788d scripts/dl_cleanup: add support for subdirectories
Allow comparing subdirectories exactly like files.

Handle a corner case where the new subdirectory
has the same tarball inside of it
as the one that was downloaded
before a subdirectory for that package was established.

Signed-off-by: Michael Pratt <mcpratt@pm.me>
2022-09-22 16:48:36 -04:00
Tianling Shen
165b0899d4
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-09-16 15:32:20 +08:00
Ansuel Smith
cf2c9498be
scripts: add clean of build dir to dl_cleanup script
Improve dl_cleanup by adding an option to also clean the build directory
related to the downloaded package.
The script will check every directory in build_dir/ and check if any
old package is present there. If outdated package are found, the old
one are cleared leaving only the last one.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2022-09-14 01:34:47 +02:00
Ansuel Smith
69760d415d
scripts: assume dl/ the default dir for dl_cleanup script
Assume dl/ the default dl dir and make it configurable if someone have that in
a different place.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2022-09-14 01:34:47 +02:00
Ansuel Smith
428c5bf3d1
scripts: add additional regex for dl_cleanup
Add additional regex for dl_cleanup script to handle
case with xxx-v1.2a and xxx-v1.2.3a

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2022-09-14 01:34:43 +02:00
Petr Štetiar
50a48faa1b scripts/download.pl: fix downloads with wget
Several users of wget for downloads (curl is not available in the
system) have reported broken download functionality:

 wget --tries=5 --timeout=20 --output-document=-  https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.10.142.tar.xz
 http://: Invalid host name.

Thats all happening due to '' was passed as an argument, which got later
expanded to http://.

In the context of a list constructor '' is not nothing, it is an empty
string element.  So fix it by using () as it will yield "nothing" and
thus not introduce an empty string element.

Fixes: #10692
Fixes: 90c6e3aedf16 ("scripts: always check certificates")
Signed-off-by: Jo-Philipp Wich <jo@mein.io> [shellwords() -> ()]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2022-09-13 09:14:09 +02:00
Petr Štetiar
c836ca84e8 scripts/download.pl: silence can't exec curl warning
When running build in verbose mode `make V=s` we can see a lot of
following warnings when curl is not available in the system:

 Can't exec "curl": No such file or directory at scripts/download.pl line 77.

So lets fix it by redirecting of the stderr to null hole.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2022-09-13 09:14:09 +02:00
Josh Roys
90c6e3aedf
scripts: always check certificates
Remove flags from wget and curl instructing them to ignore bad server
certificates. Although other mechanisms can protect against malicious
modifications of downloads, other vectors of attack may be available
to an adversary.

TLS certificate verification can be disabled by turning oof the
"Enable TLS certificate verification during package download" option
enabled by default in the "Global build settings" in "make menuconfig"

Signed-off-by: Josh Roys <roysjosh@gmail.com>
[ add additional info on how to disable this option ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-09-10 15:24:22 +02:00
Tianling Shen
090f0623cf
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-09-08 09:31:50 +08:00
Petr Štetiar
06e01e817e scripts: xxdi.pl: add xxd -i compat mode
So it can serve as a standalone drop in replacement for xxd utility used
currently mostly in U-Boot packages with `xxd -i` mode which outputs C
include file style, with aim for byte to byte identical output, so the
eventual difference in the generated output is easily spottable.

Fixes: #10555
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Signed-off-by: Jo-Philipp Wich <jo@mein.io> [perl-fu]
2022-09-06 08:04:53 +02:00