USB Creator

[Arjen Balfoort]

USB Creator is now using 7z to check the configuration inside the ISO. Mounting the ISO worked fine until maintainers create non-ISO9660 ISOs. 7z opens them nonetheless.

It checks grub.cfg or isolinux.cfg for the path of the kernel, the path of the initramfs and the boot options. For most distributions that is more than enough and this way I can limit the configuration files needed to create the Grub2 menuentry.

If the ISO does not have a working loopback.cfg, USB Creator falls back to the configuration files found at f/usr/share/usb-creator/files/distributions.
If no distribution specific configuration file exists, USB Creator falls back to the generic configuration of the distribution's parent distribution. All distribution families are stored in /usr/share/usb-creator/files/distributions/families which is created from output of the Distrowatch database.

I've tested many distributions successfully:

Code: Select all

Automatic:
* 4MLinux
* Antergos
* AntiX
* ArchBang
* ArchLabs
* ArchLinux
* ArcoLinux
* CentoOS
* ClearOS
* Clonezilla
* Debian Live
* Debian Netinstall
* Deepin
* Devuan
* Elementary
* Fedora WS Live
* Fedora WS Netinstall
* Gentoo
* GnuStep
* GParted
* Kaos
* KdeNeon
* Knoppix
* Kodachi
* Kubuntu
* Linux Lite
* Linux Mint
* Lubuntu
* MX Linux
* Netrunner
* OpenMandriva
* OpenSuse Live
* Peppermint
* Rosa
* Slackware
* Slackware Live
* SolusOS
* SolydK
* SolydX
* SystemRecueCD
* Ubuntu
* Xubuntu
* Zorin

Manually:
* Kali (loopback does not boot)
* Parrot (loopback does not boot)
* Puppy (needs unpacking first)
* TinyCore

But I couldn't get these to boot:

Code: Select all

* Alpine
* Antergos minimal
* BlackArch
* FreeBSD
* Kolibri
* Mageia
* Manjaro
* OpenSuse (DVD)
* PCLinux
* ReactOS
* Sabayon
* Vector

[troypulk]

I'm testing this now and using a console to see thats going on it has been copying the iso to the thumb drive for over 10 min. now, how long does this usually take?

I'm using solydx_10_64_201907.iso for this test.

It says that it's "Gathering ISO Information"

In the older version it would of been done by now.

Thanks

EDIT:
I remember in the old version there was a choice for a single live iso and it wouldn't work unless it was checked.

By the way it's been "Gathering ISO Information" for over 40 min.

It froze here:

Code: Select all

/usr/bin/usb-creator
/usr/lib/usb-creator/main.py:56: PyGIDeprecationWarning: Since version 3.11, calling threads_init is no longer needed. See: https://wiki.gnome.org/PyGObject/Threading
  GObject.threads_init()

(main.py:7545): Gtk-WARNING **: 11:54:02.640: Theme parsing error: gtk.css:337:24: Invalid name of pseudo-class
('Executing:', 'usb-creator  "/home/troypulk/solydx_10_64_201907.iso" /dev/sdb')
==========>>>>> Start log at Sat 06 Jul 2019 11:54:49 AM PDT <<<<<==========
Set label: fatlabel /dev/sdb1 "USBCREATOR"
0x41: Dirty bit is set. Fs was not properly unmounted and some data may be corrupt.
 Automatically removing dirty bit.
Grub files copied to /media/troypulk/D6FD-6173/boot/grub/
Gather ISO information: /home/troypulk/solydx_10_64_201907.iso
Check available space: ISOSIZE=1775392, FREESIZE=3842892
ISO label: SOLYDX_10_64BIT
Found distribution: debian/solydxk
Copy /home/troypulk/solydx_10_64_201907.iso to /dev/sdb1

[Arjen Balfoort]

Have you tried a different pen drive?
Format it with fat32 (no GPT).

For me it wouldn't work on a NTFS formatted Pen drive.

I had to remove some features because those were making the previous USB Creator buggy and impossible to maintain.

[troypulk]

I did try a different pen drive but looking at it with gparted it had GPT and fat32 but it didn't work so I changed the partition table to msdos, and formatting to ext4 and fat32 and neither worked so I'm wondering if the iso is corrupted so I'm playing around with it.

I burnt the ISO to a dvd and it worked and installed just fine so the iso isn't bad.

So far there is nothing that I have done that works, what did you do to get it to work and I'll try that.

[palimmo]

This could be possibly interesting:

yesterday I downloaded the Debian 10 Gnome 64 bit iso
https://cdimage.debian.org/debian-cd/cu ... -gnome.iso
and I burnt it with SolydK USB creator on a USB stick.
According to the dialog window, the operation was successfully.

After the bios screen the OS could sadly not be loaded. I get only a blank screen with a blinking cursor.
I can only force the laptop to reboot.

SHA256 was checked and was fine.

I'll try again (downloading and burning) and I'll let you know.

[Arjen Balfoort]

After using the USB Creator, check the log: ~/.usbcreator/usbcreator.log
Perhaps there's an error logged that helps pointing in the right direction.

I did what troypulk did: msdos, fat32 and run USB Creator and I did that with all the distributions mentioned on Github.
.

[Arjen Balfoort]

I tried to create a bootable USB on my old laptop of SolydK9 and SolydK10 ISOs.
They boot but it ends in a mouse pointer for SolydK9 and a non-blinking cursor for SolydK10.
That's because this mini laptop doesn't have the resources to live boot from USB.

If somebody can confirm that the SolydXK 10 ISOs are unboatable I'll remove them from the server.
I can build new ones in a couple of weeks.

[grizzler]

The ISOs themselves are probably fine. I just wrote SolydX 10 to a USB stick and it booted without issues.

Mind you, I used UNetbootin to write the ISO to the stick and the machine I booted it on is an old BIOS-laptop...

[troypulk]

I tried to create a bootable USB on my old laptop of SolydK9 and SolydK10 ISOs.
They boot but it ends in a mouse pointer for SolydK9 and a non-blinking cursor for SolydK10.
That's because this mini laptop doesn't have the resources to live boot from USB.

If somebody can confirm that the SolydXK 10 ISOs are unboatable I'll remove them from the server.
I can build new ones in a couple of weeks.

I burned solydx_10_64_201907.iso to a DVD and it booted just fine, it's probably the usb-creator or human error.

[troypulk]

After using the USB Creator, check the log: ~/.usbcreator/usbcreator.log
Perhaps there's an error logged that helps pointing in the right direction.

I did what troypulk did: msdos, fat32 and run USB Creator and I did that with all the distributions mentioned on Github.
.

It froze after copying the iso but log didn't have anything just what it was doing.

[Arjen Balfoort]

If Grub was installed correctly, there must be something wrong with grub.cfg.

Now that I think of it. USB Creator installs 64-bit EFI Grub, 32-bit EFI Grub and legacy Grub on the USB. Perhaps that's a bit too much?

[ilu]

USB Creator 0.7.4 still hangs after writing the ISO - now with the message "Gather ISO information". Last log message is about copying.

Also USB creator mounts all available partitions. It asked for permissions on the first run - which was denied - but mounted without permission on subsequent runs.

[Arjen Balfoort]

Can you still attach the log (ISO and USB stick info)?
Did you run in VB or on metal?

Also USB creator mounts all available partitions. It asked for permissions on the first run - which was denied - but mounted without permission on subsequent runs.

I don't understand. You started usb-creator and it asked for your password which you declined but still the usb-creator was started?

Hmmm, I think I understand. usb-creator asks for a password not to mount the usb but for the tasks needed after that: installing grub to the USB, for example. It should not start at all if you decline. Don't know how to solve that one but I think it is no deal breaker for the release of the ISOs.

On the other hand, usb-creator asked me for a password after that if it needed one.

[EDIT]
I have seen this behavior on SolydX in VB. I haven't tried on metal, yet. However, I see that rsync is running by checking the USB:

Code: Select all

ls -al /media/solydxk/USBCREATOR | grep iso

It is just not updating the GUI.

[ilu]

Can you still attach the log (ISO and USB stick info)?
Did you run in VB or on metal?

Metal

I don't understand. You started usb-creator and it asked for your password which you declined but still the usb-creator was started?

It asked for (sudo) password and I entered it. So far everything is ok.
Then it asked for the password to mount - one after the other - ALL available partitions. (I don't do user-auto-mount except for fstab.) I declined for all of them. It did not mount anything except the USB. So far ok.
On second run it asked for the password - so far ok. Then - without asking - it mounted everything - also those partitions that are not mounted for a reason. Since USB creator never finishes of course nothing gets unmounted.

Hmmm, I think I understand. usb-creator asks for a password not to mount the usb but for the tasks needed after that: installing grub to the USB, for example. It should not start at all if you decline. Don't know how to solve that one but I think it is no deal breaker for the release of the ISOs.

Yes no dealbreaker. I just noticed it. Sorry for being unclear.

The log looked different immediately after running - no idea when it changed. There's no info about the pendrive in it.

Code: Select all

log                           DEBUG     ISO on /media/.../USBCREATOR: solydx_10_64_201908.iso, 1616 MB, /usr/lib/usb-creator/../../share/usb-creator/files/grub/themes/usb-creator/icons/solydxk.png
log                           DEBUG     ISO on /media/.../USBCREATOR: solydx_9_64_201902.iso, 1590 MB, /usr/lib/usb-creator/../../share/usb-creator/files/grub/themes/usb-creator/icons/solydxk.png
log                           DEBUG     Selected device info: {'available': 112960704.0, 'new_iso_required': 0, 'size': 122581608.0, 'mount': '/media/..../USBCREATOR', 'new_iso': '', 'path': '/dev/sdj'}
log                           DEBUG     ISO on /media/..../USBCREATOR: solydx_10_64_201908.iso, 1616 MB, /usr/lib/usb-creator/../../share/usb-creator/files/grub/themes/usb-creator/icons/solydxk.png
log                           DEBUG     ISO on /media/.../USBCREATOR: solydx_9_64_201902.iso, 1590 MB, /usr/lib/usb-creator/../../share/usb-creator/files/grub/themes/usb-creator/icons/solydxk.png
log                           DEBUG     Selected device info: {'available': 112960704.0, 'new_iso_required': 0, 'size': 122581608.0, 'mount': '/media/.../USBCREATOR', 'new_iso': '', 'path': '/dev/sdj'}
log                           DEBUG     >>> Start USB Creator: 2019-08-15 13:58:38.644506 <<<

Just fyi , the stick has 110 GB ...

[Arjen Balfoort]

What does this return:

Code: Select all

lsblk -o +FSTYPE /dev/sdX

Where sdX is your USB device.

I need to try to recreate the situation before I can debug.

[ilu]

Code: Select all

lsblk -o +FSTYPE /dev/sdj
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT            FSTYPE
sdj      8:144  1 119,3G  0 disk                       
├─sdj1   8:145  1     2M  0 part                       
├─sdj2   8:146  1    17M  0 part /media/.../ESP        vfat
└─sdj3   8:147  1 119,3G  0 part /media/.../USBCREATOR ext4

[Arjen Balfoort]

So, standard it is...I'll check if I have the time.

[ilu]

I'm not sure anymore ... I forgot it was running and 2 hours later its now "Installing grub". So maybe it doesn't crash but just ... looks at every byte from every possible angle? My pen drive is admittedly really big but at 10 Euro it's a common size nowadays.. I tried to understand your code but I'm still useless as ever.

Edit: +6h and still "Install Grub"

Edit: On the second day USB creator only asks for mounting permissions - maybe it did the same yesterday and I just did not watch carefull enough? - and only for 2 partitions (out of five) . I declined as always, they were mounted anyway. This time I chose "write single ISO file" - whatever that means - and USB creator reported "copying ISO" but in fact nothing happened on the pen drive. Instead USB creator again asked for mounting permissions on those 2 partitions which I again declined. I clicked "details" and there were 2 links to the udisks project sites, so I'm guessing udisks somehow messes things up here. After some time - relatively quickly - USB creator reported success but in reality nothing happened and nothing was written.

Next try I did not tick that "write single ISO file" checkbox and things went as usual. File was written and USB creator hang up on "Install Grub ...". I noticed that the execute permissions on the copied ISO file were not set so maybe that gives you a hint were things go wrong.

Just in case this helps: After copying 7 ISO files only 2 of them made it into the grub.cfg. Sadly there is no repair function anymore which always fixed things in the old version.

Code: Select all

# USB Creator grub.cfg

insmod font
if loadfont /boot/grub/themes/usb-creator/unifont-16.pf2; then
    if keystatus --shift ; then true ; else
        if [ x"${grub_platform}" = xefi ]; then
            insmod efi_gop
            insmod efi_uga
            insmod videotest
            insmod videoinfo
        else
            insmod vbe
            insmod vga
        fi
        insmod loopback
        insmod iso9660
        insmod ufs2
        insmod ntfs
        insmod bsd
        insmod geli
        insmod gzio
        insmod part_msdos
        insmod part_gpt
        insmod all_video
        insmod gfxterm
        insmod gfxmenu
        set gfxmode=auto
        set gfxpayload=auto
        terminal_output gfxterm
        if terminal_output gfxterm ; then true ; else
            terminal gfxterm
        fi
        # Load theme
        insmod png
        loadfont /boot/grub/themes/usb-creator/unifont-20.pf2
        set theme=/boot/grub/themes/usb-creator/theme.txt
        export theme
    fi
fi

# Set variables
set timeout=10
set default=0

menuentry 'Solydxee 10 64-bit 201902' --class solydxk {
    set iso_path='/solydxee_10_64_201902.iso'
    loopback loop $iso_path
    linux (loop)/live/vmlinuz findiso=$iso_path 	 boot=live config quiet splash 
    initrd (loop)/live/initrd.img 
}

menuentry 'Solydk 10 64-bit 201908' --class solydxk {
    iso_path='/solydk_10_64_201908.iso'
    search --set -f $iso_path
    probe --set=rootuuid -u $root
    export iso_path rootuuid
    loopback loop $iso_path
    root=(loop)
    configfile /boot/grub/loopback.cfg
    loopback --delete loop
}

Sorry about this really messy bug report, I'm having problems to sort things out.

[Arjen Balfoort]

I think it is fixed in the new version: not asking for passwords for mounting fixed partitions.