182 Commits

Author SHA1 Message Date
Tianling Shen
8c8ba72a11
download.pl: add ImmortalWrt fallback
Fixes: #759

Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit a19080494926ad523e636d7e075717b69e2f3e47)
2022-08-17 20:43:30 +08:00
Tianling Shen
3a1aeb313e
download.pl: remove broken mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 9fbf1751eced799a4320fed8095b78fbde2d5116)
2022-08-06 18:50:25 +08:00
Tianling Shen
3ded6f27ad
scripts/download.pl: drop broken mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 02edeb14690c14bd7021b4be2270286bb4bb9108)
2022-05-06 22:11:33 +08:00
Tianling Shen
062547c724
scripts/download.pl: add CN mirrors for DEBIAN
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 0ea2f5715818ce637721b5c793e5cd985d0200b1)
2022-04-14 01:07:38 +08:00
Daniel Golle
cf0dfc2dad
scripts/gen_image_generic.sh: generate reproducible EFI filesystem
Generate FAT filesystem for EFI boot in a reproducible way:
 * use '--invariant' option of mkfs.fat
 * set timestamps of all files to SOURCE_DATE_EPOCH
 * make sure files are ordered locale-independent

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
(cherry picked from commit aece8f5ae8d15e5e79b8e34a176895209709afb6)
2022-04-14 01:04:34 +08:00
Paul Spooren
5bf95283f9
build: store source_date_epoch as integer
The value is retreived from a env variable which defaults to be read as
a string. However the SOURCE_DATE_EPOCH is a unix timestamp aka integer.

Fix this to allow downstream tools to parse the value directly.

Signed-off-by: Paul Spooren <mail@aparcar.org>
(cherry picked from commit 2120cad38d6f78cc1b727459eeedb6ed9a14edbe)
2022-04-12 02:45:40 +08:00
Daniel Golle
e861b2a4dd
scripts/gen_image_generic.sh: make ext4 bootfs reproducible
Set fixed timestamp for kernel other files in /boot filesystem.
This should help making x86 *combined* images reproducible.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
(cherry picked from commit 068ea2cde04e2666fb05731cea941bccfaeddc25)
2022-04-12 02:44:35 +08:00
Paul Spooren
794eb1fe61 scripts: make sure conffiles are sorted
It may happen that conffiles are in different order on different builds.
Make sure they have the same order by sorting them.

FIX: #9612

Signed-off-by: Paul Spooren <mail@aparcar.org>
2022-04-04 17:20:29 +08:00
Tianling Shen
8e5662d5a3
scripts/download.pl: drop cqu mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 7807fb01194e29936cdac7cb84790624c3f4cc81)
2022-03-17 20:00:16 +08:00
Georgi Valkov
c668329a78 mkhash: fix build errors on FreeBSD 13.0
be64enc, be16dec, and be32dec are declared on FreeBSD 13.0, in
/usr/include/sys/endian.h so we should not declare them.

Fixes the following error during feeds update:
staging_dir/host/bin/mkhash: No such file or directory

gcc scripts/mkhash.c
scripts/mkhash.c:111:1: error: redefinition of 'be64enc'
  111 | be64enc(void *buf, uint64_t u)

Signed-off-by: Georgi Valkov <gvalkov@abv.bg>
2022-03-06 23:56:37 +08:00
Eneas U de Queiroz
d88e607184 scripts/diffconfig.sh: ensure config/conf is built
diffconfig.sh runs ./scripts/config/conf, but it does not get built
with 'make {menu,x,n}config.  Call 'make ./scripts/config/conf' to
ensure it's been built before running it, aborting in case of failure.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>[removed Fixes: due revert]
2022-02-28 16:11:21 +08:00
Petr Štetiar
80e3983b36 Revert "scripts/diffconfig.sh: ensure config/conf is built"
This reverts commit c0849c1d9c17ba96a37b67363b5551c065e9f50d as it seems
to introduce regression in config.buildinfo files produced at least by
buildbots:

 $ curl -s https://downloads.openwrt.org/snapshots/targets/ath79/generic/config.buildinfo | head -2
 make[3]: Entering directory '/builder/shared-workdir/build'
 make[3]: Leaving directory '/builder/shared-workdir/build'

References: https://github.com/openwrt/openwrt/issues/9297#issuecomment-1049719381
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2022-02-26 11:16:43 +08:00
Eneas U de Queiroz
a8a70464fa scripts/diffconfig.sh: ensure config/conf is built
diffconfig.sh runs ./scripts/config/conf, but it does not get built
with 'make {menu,x,n}config.  Call 'make ./scripts/config/conf' to
ensure it's been built befpre running it.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2022-02-24 11:36:25 +08:00
Eneas U de Queiroz
29cb743c4a build: scripts/config - update to kconfig-v5.14
Functional Changes
---------- -------
- make 'imply' not impose any restrictions: allow symbols implied by y
  to become m
- change "modules" from sub-option to first-level attribute

Bugfixes
--------
- nconf: fix core dump when searching in empty menu
- nconf: stop endless search loops
- xconfig: fix content of the main widget
- xconfig: fix support for the split view mode

Other Changes
----- -------
- highlight xconfig 'comment' lines with '***'
- xconfig: navigate menus on hyperlinks
- xconfig: drop support for Qt4
- improve host ncurses detection

Update the 'option modules' usage to just 'modules' in Config.in.

Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
2022-02-24 11:35:03 +08:00
Paul Spooren
176e1fe191
scripts: dump-target-info print kernel versions
This commits adds the ability to print Kernel versions of all
targets/subtargets. If a testing Kernel is set print that version as
well.

Example output:

    apm821xx/nand 5.10
    apm821xx/sata 5.10
    arc770/generic 5.4
    archs38/generic 5.4
    armvirt/32 5.10
    armvirt/64 5.10
    at91/sam9x 5.10
    at91/sama5 5.10
    ath25/generic 5.4
    ath79/generic 5.4 5.10
    ath79/mikrotik 5.4 5.10
    --- %< ---

This should help to get a quick update on the state of Kernels.

Signed-off-by: Paul Spooren <mail@aparcar.org>
(cherry picked from commit 02de391b086dd2b7a72c2394cfb66cec666a51c1)
2022-01-01 17:26:52 +08:00
Tianling Shen
d3c10b6224
scripts/download.pl: add new mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 13c56d272d4218a706c986ef37de5be020aeafd6)
2021-11-30 20:44:49 +08:00
Tianling Shen
4c3a0af01c
scripts/download.pl: tweak default mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-11-29 13:23:45 +08:00
Tianling Shen
923d73b691
scripts/download.pl: reorder self-hosted mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit d26b4ff4de28af014b55a1040d5dd7fe01c026fb)
2021-11-29 03:31:03 +08:00
Tianling Shen
404f8f2992
scripts/download.pl: add immortalwrt's mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-11-21 01:07:21 +08:00
Felix Fietkau
cb8dc7b14e toolchain: add a version that can be bumped to force toolchain/target rebuild
This can be used to ensure that the compiled code is up to date, when
something important changes in the toolchain.
A recent example of this is the gcc 11 fix for a code miscompilation issue

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2021-11-19 23:58:57 +08:00
Daniel Golle
aa58b28e47
base-files, metadata: support additional group membership
Some packages may require additional group membership for the system
user added by that package. Allow defining additional groups as third
member of the ':'-separated tuple, allowing to specify multiple
','-separated groups with optional GID.

Example:
USERID:=foouser=1000:foogroup=1000:addg1=1001,addg2=1002,addg3

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
(cherry picked from commit b2aca61360c33ad3161f8ef35a1c28ebb2507600)
2021-11-10 15:42:57 +08:00
Josh Soref
e72185a4a1 scripts: fix various typos
This only affects typos in comments or user-facing output.

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
[only picks changes to scripts, drop "commandline" replacement,
 fix case for "arbitrary", improve commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2021-11-02 23:49:21 +08:00
Jan-Niklas Burfeind
cffc1cd77b scripts: eva_ramboot.py: remove unused import
concludes:
commit e7bc8984d9ca ("scripts: make eva_ramboot.py offset configurable")

Signed-off-by: Jan-Niklas Burfeind <git@aiyionpri.me>
2021-11-02 23:46:05 +08:00
Petr Štetiar
dc89ad3dfe
scripts/gen_image_generic.sh: fix more shellcheck warnings
Fixes following shellcheck warnings:

 In scripts/gen_image_generic.sh line 20:
 cyl=$(( (KERNELSIZE + ROOTFSSIZE) * 1024 * 1024 / (head * sect * 512)))
 ^-^ SC2034: cyl appears unused. Verify use (or export if used externally).
 --
 In scripts/gen_image_generic.sh line 34:
     [ -n "$PADDING" ] && dd if=/dev/zero of="$OUTPUT" bs=512 seek="$(($ROOTFSOFFSET + $ROOTFSSIZE))" conv=notrunc count="$sect"
                                                                       ^-----------^ SC2004: $/${} is unnecessary on arithmetic variables.
 --
 In scripts/gen_image_generic.sh line 35:
     mkfs.fat -n kernel -C "$OUTPUT.kernel" -S 512 "$(($KERNELSIZE / 1024))"
                                                       ^---------^ SC2004: $/${} is unnecessary on arithmetic variables.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
(cherry picked from commit 92d7cedcad73ff9845ea3f635ea82bd925078c1f)
2021-10-14 21:25:07 +08:00
Paul Spooren
7218521845 scripts: package-metadata add pkgmanifestjson call
The new `pkgmanifestjson` call prints all package manifest of a feed in
JSON format. This function can be used to print an overview of packages
information used for downstream tooling.

The script is entirely based on Petrs work on dependency visualisation.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-10-12 23:56:16 +08:00
Paul Spooren
736c627160 scripts: store maintainer in package metadata
The maintainer could be usable for downstream tooling, so start storing
it in the metadata.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-10-12 23:55:51 +08:00
Ansuel Smith
2969a245ec scripts: handle gcc and linux in dl_cleanup script
Handle gcc and linux with a special regex that set their progname with
their major version. This way every minor version can be cleared. The
build cleanup logic can be tweaked later to clean the entire toolchain
and target dir with a different gcc version.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
[reformat commit message]
Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-10-12 23:51:27 +08:00
Ansuel Smith
7b630b974e scripts: format dl_cleanup to black format python style
Forma dl_cleanup python script to black style.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2021-10-12 23:51:14 +08:00
Ansuel Smith
6477cae7a1 scripts: add missing regex for dl_cleanup script
Regex xxx-YYYY-MM-DD-GIT_SHASUM was missing. Add the new regex to improve
and better find outdated package. This also fix a bug where some bug were
incorrectly detected as packagename-yyyy-mm-dd instead of packagename due
to them be parsed by the wrong parser

Example:

    openwrt-keyring-2021-02-20-49283916.tar.xz

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
[added example in commit message]
Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-10-12 23:50:56 +08:00
Paul Spooren
e9cb0a7753
build: store artifacts in JSON
Multiple profiles create artifacts, these should be stored in the JSON
file as well, allowing downstream tooling to show those files, too.

Artifacts don't have specific filesystems so only the fields `name`,
`type` and `sha256` are available.

Rename env variable names from IMAGE_ to FILE_ prefixes to reflect that
images, kernels and artifacts are added with the same command.

Signed-off-by: Paul Spooren <mail@aparcar.org>
(cherry picked from commit 5ef4608c02ef6db9698c8d10011b9c14a8ca073b)
2021-09-17 04:49:04 +08:00
Adrian Schmutzler
7984a95946
treewide: rename IMAGE_PREFIX/IMAGE_NAME to DEVICE_IMG_*
We so far had two variables IMG_PREFIX and IMAGE_PREFIX with
different content. Since these names are obviously quite
confusing, this patch renames the latter to DEVICE_IMG_PREFIX,
as it's a device-dependent variable, while IMG_PREFIX is only
(sub)target-dependent.

For consistency, also rename IMAGE_NAME to DEVICE_IMG_NAME, as
that's a device-dependent variable as well.

Cc: Paul Spooren <mail@aparcar.org>
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
(cherry picked from commit ef2cb8572b48f1e1964b4d1d014d16cb721b5175)
2021-09-17 04:48:26 +08:00
SuLingGG
e45c211c5a
scripts/download.pl: update openwrt.cc mirror url
[updated url]
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 009c17217107834cdf5ccc336e791ef0b33eada1)
2021-08-30 23:07:46 +08:00
Tianling Shen
819db9dc2b
scripts/download.pl: re-add aliyun mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-07-15 22:40:00 +08:00
Ansuel Smith
8699f787e0 scripts: check if dl directory exist in dl_cleanup script
Check if the provided dl directory exist and return on error.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2021-07-06 23:06:50 +08:00
Paul Spooren
a481251f47
build,json: fix generation with empty profiles
If the image generation doesn't add any profiles to the output the
*profile merge* will fail. To avoid that set an empty profile as
fallback.

Signed-off-by: Paul Spooren <mail@aparcar.org>
(cherry picked from commit fd0d9909bf50f114d5e7f7cedf53e542de878a2c)
2021-06-21 20:35:32 +08:00
Tianling Shen
a3325e5051
download.pl: add some mirrors disabled by default
To avoid abuse, these mirrors will not be used until you enable them
manually.

Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-06-19 18:30:22 +08:00
AmadeusGhost
4b33d667e8 download.pl: refresh mirrors list
Since there are so many useless actions, just to catch up with
the 'new update' ...
2021-06-19 18:08:10 +08:00
AmadeusGhost
76b990208f
download.pl: add more mirrors for SourceForge
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-06-19 00:46:41 +08:00
Moritz Warning
2cf729818b
build: preserve profiles.json between builds
Keep other profiles.json content if the data belongs to the current
build version.

Also useful for the ImageBuilder, which builds for a single model each
time. Without this commit the profiles.json would only contain the
latest build profile information.

Signed-off-by: Moritz Warning <moritzwarning@web.de>
[improve commit message]
Signed-off-by: Paul Spooren <mail@aparcar.org>
(cherry picked from commit a463b96241fbc2d142982387eaed9989e201ac7a)
2021-06-17 20:53:42 +08:00
李国
9eddf65670
x86: generate EFI platform bootable images
Add EFI platform bootable images for x86 platforms. These images can
also boot from legacy BIOS platform.

EFI System Partition need to be fat12/fat16/fat32 (not need to load
filesystem drivers), so the first partition of EFI images are not ext4
filesystem any more.

GPT partition table has an alternate partition table, we did not
generate it. This may cause problems when use these images as qemu disk
(kernel can not find rootfs), we pad enough sectors will be ok.

Signed-off-by: 李国 <uxgood.org@gmail.com>
[part_magic_* refactoring, removed genisoimage checks]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
(cherry picked from commit a6b7c3e672764858fd294998406ae791f5964b4a)
2021-06-17 20:13:41 +08:00
Karel Kočí
f3e47345b1 scripts/feeds: generate index after all feeds are updated
This separates index update from feed update. The result is that all
requested feeds are first updated and only then indexed.

The reason for this change is to prevent errors being reported and
potentially invalid index being generated thanks to cross feeds
dependency.
The feeds script pulls in default all feeds as they come and on install
prefers packages from first feeds (unless special feed is requested).
Thus order of feeds in some way specifies preferences. This is handy for
downstream distributions as they can simply override any package from
upstream feeds by placing their feed before them. This removes need to
patch or fork upstream feeds.
The problem is that such feed most likely depends in some way also on
subsequent feeds. The most likely feeds are 'packages' or 'luci'. The
example would be Python package that needs 'python.mk' from 'packages'
feed. Ordering custom feed after dependent feeds is sometimes just not
possible because of preference requirement described before.
The solution is to just first pull all feeds and generate indexes only
after that. In the end this ensures that index is generated correctly at
first try without any error.

In terms of code this removes 'perform_update' argument from
'update_feed' as with index update removal the update is the only action
performed in that subroutine. Thus this moves condition to 'update'
subroutine.

Signed-off-by: Karel Kočí <karel.koci@nic.cz>
2021-06-08 23:16:09 +08:00
Paul Spooren
2c11e3e346 scripts: config.guess: update to 2021-05-24
This script hasn't seen an update in multiple years, update it to the
latest version provided upstream. Both `config.guess` and `config.sub`
are copied from upstream[1] and not modified.

The full changelog is available within the upstream repository[1].

[1]: https://git.savannah.gnu.org/git/config.git

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-05-29 23:21:13 +08:00
Tianling Shen
f6ef5a895c
scripts/download: update mirror url
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-05-26 22:17:21 +08:00
Leonardo Mörlein
2a821d862c
build: introduce $(MKHASH)
Before this commit, it was assumed that mkhash is in the PATH. While
this was fine for the normal build workflow, this led to some issues if

    make TOPDIR="$(pwd)" -C "$pkgdir" compile

was called manually. In most of the cases, I just saw warnings like this:

    make: Entering directory '/home/.../package/gluon-status-page'
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    [...]

While these were only warnings and the package still compiled sucessfully,
I also observed that some package even fail to build because of this.

After applying this commit, the variable $(MKHASH) is introduced. This
variable points to $(STAGING_DIR_HOST)/bin/mkhash, which is always the
correct path.

Signed-off-by: Leonardo Mörlein <me@irrelefant.net>
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-05-13 22:49:42 +08:00
Tianling Shen
1b76d7006b
scripts/download.pl: only call our own mirror when failed to download
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-04-19 01:36:37 +08:00
Tianling Shen
29b53e375a
scripts/download.pl: add aliyun(cn) mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-04-19 00:15:44 +08:00
Tianling Shen
9961791b9e
scripts/download.pl: remove unavailable mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-04-17 00:12:02 +08:00
Tianling Shen
4f53a26f06
scripts/download.pl: update mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-04-13 21:33:25 +08:00
Tianling Shen
3364f9a63d
scripts/download.pl: re-order mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-26 22:16:38 +08:00
Paul Spooren
5ec2e9b8c7 build,json: 3rd fixup of default_packages
This became a bit of a tragedy, caused by a corner cases which wasn't
put into account during testing. DEFAULT_PACKAGES are defined in
target/linux/<target>/Makefile but also in
target/linux/<target>/<subtarget>/target.mk.

The latter was no longer imported when using DUMP=1, however not using
DUMP=1 while running the Makefile in target/linux/<target>/ caused duplicate
packages in the list.

As a solution, which should have been used from day 0, `make` runs in
target/linux/ without DUMP=1, resulting in no duplicate packages and all
inclusions from include/target.mk, linux/target/<target>/{Makefile,
<subtarget>/target.mk}

While at it, sort the list of default packages.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-26 20:00:19 +08:00