bls
Posts: 2308
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA

SD Card Image Manager

Mon May 04, 2020 9:55 pm

Like many of you, I have a small fleet of Raspberry Pi devices now, and when I rebuild them, I want them all set up "just so", with the configuration, apps, and personal files all in their places. Identically, on each of the systems.

And, like some (many?) of you, I have accumulated a set of scripts over time to set up a new system. I just finished a tooling overhaul, which resulted in sdm, an SD Card Image Manager tool.

sdm lets you:
  • Easily build and maintain a Raspbian "golden image" (.IMG file) with all of your customizations, keymaps, locales, timezones, and favorite apps installed in it.
  • Write that shiny golden image to an SD Card with the host name of your choice in it such that it's ready to go.
  • Do these things quickly and easily with minimal think or wait time.
  • Easily include your own configuration scripts
Ready-to-go means that your system has all your personal customizations installed and all Raspbian packages and updates installed at first system boot. If you're building a new fleet of Pi systems, you can apt update the golden image, and each of the new Pis will boot up fully updated.

With sdm you'll spend a lot less time rebuilding SD cards, configuring your system, and installing packages, and more time on the things you really want to do with your Pi.

As a special bonus, sdm includes a configuration script for apt-cacher-ng (client and server). apt-cacher-ng enables you to update all your Pis quickly by caching local packages. This reduces install and update time, and can greatly reduce network consumption. And, it's cooler than a ginsu knife. :lol:

sdm only runs on Raspbian, and requires a USB SD Card reader for writing a new SD Card. No, you cannot use sdm to rewrite the running system's SD Card.

I know that there are several other tools like this around. I opted to write my own in the process of upgrading my tooling. It's super-simple to install and use, and has no external dependencies, other than systemd-nspawn, which you must install via

Code: Select all

 sudo apt install systemd-container
Read about it and grab it from https://github.com/gitbls/sdm. Or, if you want to just grab the files, download the EZ downloader at https://raw.githubusercontent.com/gitbl ... mInstaller

Hope you find it useful. Feedback welcome, of course.
Pi tools:
Quickly and easily build customized-just-for-you SSDs/SD Cards: https://github.com/gitbls/sdm
Easily run and manage your network's DHCP/DNS servers on a Pi: https://github.com/gitbls/ndm
Easy and secure strongSwan VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

User avatar
Botspot
Posts: 1816
Joined: Thu Jan 17, 2019 9:47 pm
Location: Texas

Re: SD Card Image Manager

Tue May 05, 2020 1:31 am

bls wrote:
Mon May 04, 2020 9:55 pm
I just finished a tooling overhaul, which resulted in sdm, an SD Card Image Manager tool.
That looks pretty useful!

BTW I've made my own image manager: Pi-Power-Tools. So I hope it's OK to ask lots of questions and give suggestions.

For example, suggestions like this one:
On line 357 of your main sdm script, why not add the -b flag to your systemd-nspawn command? That way systemctl/systemd will be running, and means you can do a lot more stuff, for example running graphical apps.
Easily install anything on your Pi using Pi-Apps! Over 1 million users.
150+ apps including image editors, text editors, games, web browsers, appearance themes, and other upgrades to get the most out of your Pi.
More info: https://github.com/Botspot/pi-apps

bls
Posts: 2308
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA

Re: SD Card Image Manager

Tue May 05, 2020 3:25 am

Botspot wrote:
Tue May 05, 2020 1:31 am
bls wrote:
Mon May 04, 2020 9:55 pm
I just finished a tooling overhaul, which resulted in sdm, an SD Card Image Manager tool.
That looks pretty useful!

BTW I've made my own image manager: Pi-Power-Tools. So I hope it's OK to ask lots of questions and give suggestions.

For example, suggestions like this one:
On line 357 of your main sdm script, why not add the -b flag to your systemd-nspawn command? That way systemctl/systemd will be running, and means you can do a lot more stuff, for example running graphical apps.
Thanks, @Botspot for taking a look at it. I'd looked at your Power Tools previously...thought they were pretty cool, but I'm not usually a GUI guy
:roll:

Definitely appreciate the feedback. I was looking at the nspawn documentation for a way to run a command and then prompt for input. Looks like the --pipe switch might do the trick, but it appears that Buster's systemd is a bit behind on that.

I'll take a look at the -b flag tomorrow and see how it works. At a minimum, I'll make it an option...easy enough to do!
Pi tools:
Quickly and easily build customized-just-for-you SSDs/SD Cards: https://github.com/gitbls/sdm
Easily run and manage your network's DHCP/DNS servers on a Pi: https://github.com/gitbls/ndm
Easy and secure strongSwan VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

User avatar
Botspot
Posts: 1816
Joined: Thu Jan 17, 2019 9:47 pm
Location: Texas

Re: SD Card Image Manager

Tue May 05, 2020 12:52 pm

bls wrote:
Tue May 05, 2020 3:25 am
Thanks, @Botspot for taking a look at it. I'd looked at your Power Tools previously...thought they were pretty cool, but I'm not usually a GUI guy
Completely understand.
Have you seen vdesktop? It's the CLI script that PPT uses to boot things. Link here: https://github.com/Botspot/vdesktop
bls wrote:
Tue May 05, 2020 3:25 am
I was looking at the nspawn documentation for a way to run a command and then prompt for input.
I'm not sure what you mean. Do you mean "I want to run a command in the container automatically"?
Or do you mean "I can't type anything in the container. Like if apt asks for [Y/n], It doesn't 'see' my keyboard input"?
Easily install anything on your Pi using Pi-Apps! Over 1 million users.
150+ apps including image editors, text editors, games, web browsers, appearance themes, and other upgrades to get the most out of your Pi.
More info: https://github.com/Botspot/pi-apps

bls
Posts: 2308
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA

Re: SD Card Image Manager

Tue May 05, 2020 1:57 pm

Botspot wrote:
Tue May 05, 2020 12:52 pm

I'm not sure what you mean. Do you mean "I want to run a command in the container automatically"?
Or do you mean "I can't type anything in the container. Like if apt asks for [Y/n], It doesn't 'see' my keyboard input"?
The former: I want to run a single command in the container immediately after it starts, and then prompt the user for further input. With this the full process can be "touchless" for the user, if they desire (no user input required).

I put this issue aside while I finished everything else, and then in my excitement to be "done" (ha! We know that never really happens!) neglected to revisit it before I posted. I revisited this last night and now have it working. So sweet!

vdesktop...looked at it a while ago and put it on my "investigate and try" list, which I apparently lost :o . I see there are a couple of things you've sorted that I need to investigate further.

Thanks for the feedback and encouragement! Will be updating sdm later today with a couple of key improvements.
Pi tools:
Quickly and easily build customized-just-for-you SSDs/SD Cards: https://github.com/gitbls/sdm
Easily run and manage your network's DHCP/DNS servers on a Pi: https://github.com/gitbls/ndm
Easy and secure strongSwan VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

bls
Posts: 2308
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA

Re: SD Card Image Manager

Tue May 05, 2020 8:48 pm

Just posted updates that are primarily cleanup and logging-related, but added a few new features:
  • sdm now runs the Phase 1 script automatically. In fact, the whole process is automatic except for typing in the Pi password
  • Added a few new switches
    • --bootconfig to uncomment and set values for /boot/config.txt parameters
    • --bootadd to add new keys/values at the end of /boot/config.txt
    • --batch which performs the customizations and exits
    • --ddsw allows you to replace the default 'dd' switches for trying different blocksizes, etc.
    • --nspawnsw allows you to specify additional switches for the systemd-nspawn command
I think it's in really good shape with this update. Now I can go play with nspawn -b and a few other things :lol:

This will be my last update here. Further updates will be appropriately posted on github at https://github.com/gitbls/sdm
Pi tools:
Quickly and easily build customized-just-for-you SSDs/SD Cards: https://github.com/gitbls/sdm
Easily run and manage your network's DHCP/DNS servers on a Pi: https://github.com/gitbls/ndm
Easy and secure strongSwan VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

bls
Posts: 2308
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA

Re: SD Card Image Manager

Thu May 21, 2020 8:52 pm

One last post. Here's a log from building an SD Card image from the base Raspbian OS image (-Lite in this case, but -Full works equally well). Build steps include:
  • Install a bunch of apps (list in the log)
  • Install the X11 windowing system with the icewm window manager and xdm display manager (wouldn't need to do this on -Full, but you might want to install some additional graphical-based applications)
  • Set hdmigroup and hdmimode in config.txt
  • Copy a bunch of personal files (all done in my personal customize script bls-sdm-customize)
  • Set the eeprom directory to 'stable'
All completed in 5 minutes on a Pi4, with a lot of help from apt-cacher-ng.

The lines with bls-sdm-customize in them are logging my personal customizations. Everything else is "standard", and under your control. Of course, the list of apps and X11 software to install (listed in a text file) is fully customizable as well.

After the IMG file was updated, I burned it to an SD card. This process, which is also logged, sets the target hostname. The log below includes this, as well as the steps taken on the first system boot.

Those of you who are time-aware might notice that the date/time is wrong on the last few lines of the log. These are added by the firstboot script, which apparently runs before Raspbian has updated the date/time. Sigh!

Code: Select all

2020-05-20 16:25:18 Configuration Started
2020-05-20 16:25:18 Command Line: /usr/local/sdm/sdm 2020-02-13-raspbian-buster-lite.img --poptions 
2020-05-20 16:25:18  firstboot,apps,xwindows --apps @bls-apps --xapps @bls-xapps --cscript bls-sdm-customize --csrc 
2020-05-20 16:25:18  /rpi --aptcache 192.168.92.4 --hdmigroup 2 --hdmimode 82 --eeprom stable --rootpwd --user bls 
2020-05-20 16:25:18  --uid 2400
2020-05-20 16:25:18 Phase 0 Starting
2020-05-20 16:25:18 Phase 0 Custom Script '/usr/local/sdm/bls-sdm-customize'
2020-05-20 16:25:18 bls-sdm-customize Phase 0 Starting
2020-05-20 16:25:18 bls-sdm-customize Phase 0 Copy wpa_supplicant.conf
2020-05-20 16:25:18 bls-sdm-customize Phase 0 Create NFS environment
2020-05-20 16:25:18 bls-sdm-customize Phase 0 Copy /usr/local/bin config scripts
2020-05-20 16:25:19 bls-sdm-customize Phase 0 Copy bls login files to /mnt/sdm/home/bls
2020-05-20 16:25:19 bls-sdm-customize Phase 0 Completed
2020-05-20 16:25:19 Phase 0 Completed
2020-05-20 16:25:19 Phase 1 Starting
2020-05-20 16:25:35 Phase 1 Custom Script '/usr/local/sdm/bls-sdm-customize'
2020-05-20 16:25:35 bls-sdm-customize Phase 1 Starting
2020-05-20 16:25:35 bls-sdm-customize Phase 1 Disable unused wpa_supplicants
2020-05-20 16:25:35 bls-sdm-customize Phase 1 Disabling wlan0
2020-05-20 16:25:35 bls-sdm-customize Phase 1 Disable rng-tools and install haveged
2020-05-20 16:25:39 bls-sdm-customize Phase 1 Switch cron to systemd
2020-05-20 16:25:39 bls-sdm-customize Phase 1 Enable systemd VNC sockets
2020-05-20 16:25:39 bls-sdm-customize Phase 1 Completed
2020-05-20 16:25:48 FirstBoot Set Locale to 'en_US.UTF-8'
2020-05-20 16:25:51 FirstBoot Set Keymap to 'us'
2020-05-20 16:25:51 FirstBoot Set Timezone to 'America/Los_Angeles'
2020-05-20 16:25:51 Application Installs Starting
2020-05-20 16:25:52 Application Installs Installing keychain vim emacs at-spi2-core dnsutils putty aptitude 
2020-05-20 16:25:52  qpdfview avahi-utils python3-pip uuid-runtime lftp ncftp tmux mc iftop nfs-kernel-server whois 
2020-05-20 16:25:52  telnet lsof iperf3 zip nmap feh links2 systemd-container zerofree
2020-05-20 16:27:07 Application Installs Completed
2020-05-20 16:27:07 XWindows Installs Starting
2020-05-20 16:27:07 XWindows Installs Installing xserver-xorg xserver-xorg-core xserver-common xterm x11-apps 
2020-05-20 16:27:07  xfonts-base xfonts-100dpi xfonts-75dpi xfonts-scalable xdm icewm icewm-themes 
2020-05-20 16:27:07  tigervnc-standalone-server chromium-browser firefox-esr xclip xcolors xcolorsel xcolmix retext
2020-05-20 16:28:15 XWindows Installs Completed
2020-05-20 16:28:59 Phase 1 Completed
2020-05-20 16:28:59 bls-sdm-customize Phase post-install Starting
2020-05-20 16:28:59 bls-sdm-customize Phase post-install Editing xdm config files
2020-05-20 16:28:59 bls-sdm-customize Phase post-install Disabling xdm on console
2020-05-20 16:28:59 bls-sdm-customize Phase post-install Completed
2020-05-20 16:28:59 Phase 1 Command Prompt
2020-05-20 16:36:18 Setting host name 'pix' onto SD Card image for sdm-firstboot
2020-05-20 16:36:19 Burn command line: /usr/local/sdm/sdm --burn /dev/sdb --host pix 
2020-05-20 16:36:19  2020-02-13-raspbian-buster-lite.img
2020-05-20 16:36:19 Created SD Card from 2020-02-13-raspbian-buster-lite.img
2020-02-13 08:10:22 FirstBoot Set Hostname to 'pix'
2020-02-13 08:10:23 FirstBoot Set WiFi Country to 'US'
2020-02-13 08:10:23 Executing custom FirstBoot script '/usr/local/sdm/thispi/1piboot/010-disable-triggerhappy.sh'
2020-02-13 08:10:26 Executing custom FirstBoot script '/usr/local/sdm/thispi/1piboot/020-ssh-switch.sh'
2020-02-13 08:10:28 Executing custom FirstBoot script '/usr/local/sdm/thispi/1piboot/030-disable-rsyslog.sh'
sdm "installation" (e.g., copying the scripts) and basic configuration won't take you very long. Adding your own personal customizations...that's a lifetime of edits, as you discover new stuff to add to your "standard" system image!

https://github.com/gitbls/sdm
Pi tools:
Quickly and easily build customized-just-for-you SSDs/SD Cards: https://github.com/gitbls/sdm
Easily run and manage your network's DHCP/DNS servers on a Pi: https://github.com/gitbls/ndm
Easy and secure strongSwan VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

bls
Posts: 2308
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA

Re: SD Card Image Manager

Sat Oct 10, 2020 11:04 pm

I lied. :lol: This is the last post. sdm just got a major upgrade.

Here's how to quickly and easily customize a RasPiOS IMG file and burn it to an SD Card in four three commands.

It's assumed that there is a freshly downloaded copy of a RasPiOS IMG file (e.g., 2020-08-20-raspios-buster-armhf-full.img or 2020-08-20-raspios-buster-armhf-lite.img) in the current directory, and that there is an SD Card in /dev/sde.
  • Install sdm and systemd-container

    Code: Select all

    sudo curl -L https://raw.githubusercontent.com/gitbls/sdm/master/EZsdmInstaller | bash
    
  • Customize the image

    Code: Select all

    sudo /usr/local/sdm/sdm 2020-08-20-raspios-buster-armhf-full.img --wpa /path/to/working/wpa_supplicant.conf --L10n --restart --noextend 
    
    sdm will copy your Localization settings (Keymap, Locale, Timezone, and WiFi Country) from the system on which it's running, and prompt for a new password for user pi. No additional packages will be installed in this example, but 'apt update' and 'apt upgrade' will be done.
  • Burn the image onto the SD Card

    Code: Select all

    sudo /usr/local/sdm/sdm --burn /dev/sde --hostname mypi1 2020-08-20-raspios-buster-armhf-full.img
    
Next, load the SD card into a Pi and power it up. The system will come up as it always does:
  • Resize the root file system and restarts automatically
  • After the system restarts it goes through a complete system startup, just as it always does on a fresh SD Card
  • Toward the end of the boot process an sdm systemd service script runs once and sets the WiFi country, unblocking WiFi
  • When the system boot is fully complete the system automatically restarts again because of the --restart command switch
When the system comes back up your Pi is all happy, ready to go, and configured with:
  • The latest RasPiOS updates installed for installed packages
  • Password set for user pi
  • Hostname set to mypi1, or whatever you choose for the hostname
  • Keymap, Locale, and Timezone configured the same as the system on which you are running sdm
  • Wifi configured and operational
  • SSH enabled
What else can sdm do? Here are a few examples:
  • Install applications - Editors (emacs, vim, etc), and any other packages you always install in a new system. sdm has two built-in package install lists, creatively named apps and xapps. You can select which of the two lists to include when you build an image, so you can build images with no additional apps, apps only, xapps only, or both.
  • Personal customizations - Have every system come up running with your own customizations such as your favorite .bashrc and any other files that you always want on your system
  • Enable Pi-specific devices - Easily enable the serial port, camera, i2c, etc, via raspi-config automation
  • Append Custom fstab file to /etc/fstab - Automatically append your site-specific fstab entries to /etc/fstab
  • systemd service configuration and management - If there are services that you always enable or disable, you can easily configure them with sdm
  • Other customizations - Done through a simple batch script. The file sdm-customphase is a skeleton Custom Phase Script that you can copy, modify, and use. (Full disclosure: You'll need to use a Custom Phase Script to copy your .bashrc or other personal files, or perform systemd service management, etc. See the section Custom Phase Script on the github for details.)
  • Burn SD Card Image for network distribution - You can build a customized SD Card Image to distribute via a mechanism other than an actual SD Card (e.g., over the Internet). The recipient can burn the SD Card using any one of a number of tools on:
https://github.com/gitbls/sdm
Pi tools:
Quickly and easily build customized-just-for-you SSDs/SD Cards: https://github.com/gitbls/sdm
Easily run and manage your network's DHCP/DNS servers on a Pi: https://github.com/gitbls/ndm
Easy and secure strongSwan VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

bls
Posts: 2308
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA

Re: SD Card Image Manager

Thu Oct 15, 2020 6:31 pm

If you have a USB Card Reader and a spare SD Card, here's how you can recover from a lost or forgotten password on your favorite SD Card using only your Pi.
  • Install a fresh copy of RasPiOS on the spare SD Card and boot it up
  • After it boots, login and install sdm. This should only take a minute or so.

    Code: Select all

    sudo curl -L https://raw.githubusercontent.com/gitbls/sdm/master/EZsdmInstaller | bash
    
  • Put your favorite SD Card in the USB Card Reader and use sdm to --explore the SD Card. Replace /dev/sdX with the actual name of your SD Card (e.g., /dev/sda)

    Code: Select all

    bash$ sudo /usr/local/sdm/sdm --explore /dev/sdX
    root@sdm:~# passwd pi
    
  • After you've changed the password, exit the sdm --explore session.
  • Shut down the spare SD Card system and remove the spare SD Card from your Pi.
  • Move your favorite SD Card from the USB reader to it's proper home in your Pi and restart the system
Done! Now you can login with your new password.

Here's an annotated complete log of using the spare SD Card system to update the password for user 'pi' on my "favorite" SD Card:

Code: Select all

pi@rpi3p2~ $
# ** We've just restarted the system and logged in. Install sdm
pi@rpi3p2~ $ sudo curl -L https://raw.githubusercontent.com/gitbls/sdm/master/EZsdmInstaller | bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1739  100  1739    0     0  34098      0 --:--:-- --:--:-- --:--:-- 34098
Downloading 'sdm' to '/usr/local/sdm/sdm'
Downloading 'sdm-phase0' to '/usr/local/sdm/sdm-phase0'
Downloading 'sdm-phase1' to '/usr/local/sdm/sdm-phase1'
Downloading 'sdm-cparse' to '/usr/local/sdm/sdm-cparse'
Downloading 'sdm-firstboot' to '/usr/local/sdm/sdm-firstboot'
Downloading 'sdm-apt-cacher' to '/usr/local/sdm/sdm-apt-cacher'
Downloading 'sdm-apt' to '/usr/local/sdm/sdm-apt'
Downloading 'sdm-customphase' to '/usr/local/sdm/sdm-customphase'
Downloading 'sdm-apps-example' to '/usr/local/sdm/sdm-apps-example'
Downloading 'sdm-xapps-example' to '/usr/local/sdm/sdm-xapps-example'
Downloading '1piboot/1piboot.conf' to '/usr/local/sdm/1piboot/1piboot.conf'
Downloading '1piboot/010-disable-triggerhappy.sh' to '/usr/local/sdm/1piboot/010-disable-triggerhappy.sh'
Downloading '1piboot/030-disable-rsyslog.sh' to '/usr/local/sdm/1piboot/030-disable-rsyslog.sh'
Checking if systemd-container is installed
 Installing systemd-container
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libcurl3-gnutls
Recommended packages:
  btrfs-progs libnss-mymachines
The following NEW packages will be installed:
  libcurl3-gnutls systemd-container
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 610 kB of archives.
After this operation, 1,410 kB of additional disk space will be used.
Get:1 http://raspbian.raspberrypi.org/raspbian buster/main armhf libcurl3-gnutls armhf 7.64.0-4+deb10u1 [292 kB]
Get:2 http://raspbian.raspberrypi.org/raspbian buster/main armhf systemd-container armhf 241-7~deb10u4+rpi1 [318 kB]
Fetched 610 kB in 0s (14.1 MB/s)       
Selecting previously unselected package libcurl3-gnutls:armhf.
(Reading database ... 60145 files and directories currently installed.)
Preparing to unpack .../libcurl3-gnutls_7.64.0-4+deb10u1_armhf.deb ...
Unpacking libcurl3-gnutls:armhf (7.64.0-4+deb10u1) ...
Selecting previously unselected package systemd-container.
Preparing to unpack .../systemd-container_241-7~deb10u4+rpi1_armhf.deb ...
Unpacking systemd-container (241-7~deb10u4+rpi1) ...
Setting up libcurl3-gnutls:armhf (7.64.0-4+deb10u1) ...
Setting up systemd-container (241-7~deb10u4+rpi1) ...
Created symlink /etc/systemd/system/multi-user.target.wants/machines.target ? /lib/systemd/system/machines.target.
Processing triggers for man-db (2.8.5-2) ...
Processing triggers for dbus (1.12.20-0+deb10u1) ...
Processing triggers for libc-bin (2.28-10+rpi1) ...

Invoke sdm with 'sudo /usr/local/sdm/sdm'
pi@rpi3p2~ $
# ** Let's make sure we get the correct device for my favorite SD Card
pi@rpi3p2~ $ sudo journalctl -f
-- Logs begin at Thu 2020-08-20 03:47:33 PDT. --
Oct 15 11:02:06 rpi3p2 sudo[1167]: pam_unix(sudo:session): session opened for user root by pi(uid=0)
Oct 15 11:02:25 rpi3p2 kernel: sd 0:0:0:0: [sda] 31116288 512-byte logical blocks: (15.9 GB/14.8 GiB)
Oct 15 11:02:30 rpi3p2 kernel:  sda: sda1 sda2
Oct 15 11:02:33 rpi3p2 kernel: sd 0:0:0:0: [sda] 31116288 512-byte logical blocks: (15.9 GB/14.8 GiB)
Oct 15 11:02:33 rpi3p2 kernel:  sda: sda1 sda2
^C
# ** Invoke sdm to --explore the SD Card using the device name we found above (/dev/sda in this case)
pi@rpi3p2~ $ sudo /usr/local/sdm/sdm --explore /dev/sda
Mounting device '/dev/sda'
mount: /dev/sda2 mounted on /mnt/sdm.
mount: /dev/sda1 mounted on /mnt/sdm/boot.

* Ready to nspawn into the device '/dev/sda'
** Type 'exit' to exit nspawn
Spawning container sdm on /mnt/sdm.
Press ^] three times within 1s to kill container.
root@sdm:~# 
# ** Check the date of /etc/shadow on the SD Card, where the encrypted  passwords are kept
root@sdm:~# ls -l /etc/shadow
-rw-r----- 1 root shadow 912 Oct 15 10:32 /etc/shadow
root@sdm:~# 
# ** Change the password for user 'pi' on the SD Card
root@sdm:~# passwd pi
New password: 
Retype new password: 
passwd: password updated successfully
root@sdm:~# ls -l /etc/shadow
-rw-r----- 1 root shadow 912 Oct 15 11:03 /etc/shadow
# ** /etc/shadow on the SD Card was indeed updated
# ** Now exit the sdm --explore session
root@sdm:~# exit
logout
Container sdm exited successfully.
umount: /mnt/sdm/boot unmounted
umount: /mnt/sdm/ unmounted
Done
pi@rpi3p2~ $
# ** But /etc/shadow on the running system's SD Card was NOT updated (as expected)
pi@rpi3p2~ $ ls -l /etc/shadow
-rw-r----- 1 root shadow 1174 Aug 20 03:47 /etc/shadow
pi@rpi3p2~ $ sudo shutdown -h now
# ** Now we can put our favorite SD Card back in the Pi, reboot, and login with the new password
Pi tools:
Quickly and easily build customized-just-for-you SSDs/SD Cards: https://github.com/gitbls/sdm
Easily run and manage your network's DHCP/DNS servers on a Pi: https://github.com/gitbls/ndm
Easy and secure strongSwan VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

Mycelium
Posts: 4
Joined: Tue Dec 22, 2020 10:07 pm

Re: SD Card Image Manager

Fri Feb 05, 2021 11:40 pm

bls wrote:
Sat Oct 10, 2020 11:04 pm
I lied. :lol: This is the last post. sdm just got a major upgrade.

Here's how to quickly and easily customize a RasPiOS IMG file and burn it to an SD Card in four three commands.

It's assumed that there is a freshly downloaded copy of a RasPiOS IMG file (e.g., 2020-08-20-raspios-buster-armhf-full.img or 2020-08-20-raspios-buster-armhf-lite.img) in the current directory, and that there is an SD Card in /dev/sde.
  • Install sdm and systemd-container

    Code: Select all

    sudo curl -L https://raw.githubusercontent.com/gitbls/sdm/master/EZsdmInstaller | bash
    
  • Customize the image

    Code: Select all

    sudo /usr/local/sdm/sdm 2020-08-20-raspios-buster-armhf-full.img --wpa /path/to/working/wpa_supplicant.conf --L10n --restart --noextend 
    
    sdm will copy your Localization settings (Keymap, Locale, Timezone, and WiFi Country) from the system on which it's running, and prompt for a new password for user pi. No additional packages will be installed in this example, but 'apt update' and 'apt upgrade' will be done.
  • Burn the image onto the SD Card

    Code: Select all

    sudo /usr/local/sdm/sdm --burn /dev/sde --hostname mypi1 2020-08-20-raspios-buster-armhf-full.img
    
Next, load the SD card into a Pi and power it up. The system will come up as it always does:
  • Resize the root file system and restarts automatically
  • After the system restarts it goes through a complete system startup, just as it always does on a fresh SD Card
  • Toward the end of the boot process an sdm systemd service script runs once and sets the WiFi country, unblocking WiFi
  • When the system boot is fully complete the system automatically restarts again because of the --restart command switch
When the system comes back up your Pi is all happy, ready to go, and configured with:
  • The latest RasPiOS updates installed for installed packages
  • Password set for user pi
  • Hostname set to mypi1, or whatever you choose for the hostname
  • Keymap, Locale, and Timezone configured the same as the system on which you are running sdm
  • Wifi configured and operational
  • SSH enabled
What else can sdm do? Here are a few examples:
  • Install applications - Editors (emacs, vim, etc), and any other packages you always install in a new system. sdm has two built-in package install lists, creatively named apps and xapps. You can select which of the two lists to include when you build an image, so you can build images with no additional apps, apps only, xapps only, or both.
  • Personal customizations - Have every system come up running with your own customizations such as your favorite .bashrc and any other files that you always want on your system
  • Enable Pi-specific devices - Easily enable the serial port, camera, i2c, etc, via raspi-config automation
  • Append Custom fstab file to /etc/fstab - Automatically append your site-specific fstab entries to /etc/fstab
  • systemd service configuration and management - If there are services that you always enable or disable, you can easily configure them with sdm
  • Other customizations - Done through a simple batch script. The file sdm-customphase is a skeleton Custom Phase Script that you can copy, modify, and use. (Full disclosure: You'll need to use a Custom Phase Script to copy your .bashrc or other personal files, or perform systemd service management, etc. See the section Custom Phase Script on the github for details.)
  • Burn SD Card Image for network distribution - You can build a customized SD Card Image to distribute via a mechanism other than an actual SD Card (e.g., over the Internet). The recipient can burn the SD Card using any one of a number of tools on:
https://github.com/gitbls/sdm
Hey,

This sounds amazing and I think just what I want/need. I've been basically taking notes of the steps needed to replicate each installation (I'm going to be installing same system for about 5-10 Raspberries in the next 2-3 months). I've never used any of this though, so I'm just reading about how to use it on Github.

I'll let you know how it goes, thank you for sharing.

bls
Posts: 2308
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA

Re: SD Card Image Manager

Sat Feb 06, 2021 3:22 pm

Mycelium wrote:
Fri Feb 05, 2021 11:40 pm

This sounds amazing and I think just what I want/need. I've been basically taking notes of the steps needed to replicate each installation (I'm going to be installing same system for about 5-10 Raspberries in the next 2-3 months). I've never used any of this though, so I'm just reading about how to use it on Github.

I'll let you know how it goes, thank you for sharing.
Great that you found it interesting. Do let me know if you have ANY questions or problems, either here or on the github.

I'm really excited about sdm because it makes it SO easy to build new SSD/SD Cards with EXACTLY the configuration I want, without having to remember all the steps, and my Pi(s) come up fully configured and ready for me to use, rather than ready for me to wrestle with configuration steps.

Hope you find that it fits your needs, too!
Pi tools:
Quickly and easily build customized-just-for-you SSDs/SD Cards: https://github.com/gitbls/sdm
Easily run and manage your network's DHCP/DNS servers on a Pi: https://github.com/gitbls/ndm
Easy and secure strongSwan VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

bls
Posts: 2308
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA

Re: SD Card Image Manager

Sat Feb 20, 2021 7:11 pm

V4.02 is available and has the ability to configure Virtual VNC desktops.

They play well with RealVNC on the Console. While RealVNC Server is already installed on RasPiOS Desktop, it's not in RasPiOS Lite by default, so sdm can install that as well.

Here's a shot of 3 desktops, all running on the same Pi4. The large desktop is the Pi4 console. One of the Virtual Desktops is using RealVNC Viewer, and one is using TightVNC Viewer.

Image
Pi tools:
Quickly and easily build customized-just-for-you SSDs/SD Cards: https://github.com/gitbls/sdm
Easily run and manage your network's DHCP/DNS servers on a Pi: https://github.com/gitbls/ndm
Easy and secure strongSwan VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

bls
Posts: 2308
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA

Re: SD Card Image Manager

Mon Apr 12, 2021 11:39 pm

V4.06 is available with 2 handy new switches specifically for LXDE desktops.

Code: Select all

-- mouse left    Sets the mouse for the primary user (pi, or the user you specified) to be left-handed
--nopiwiz        Disable piwiz automatic start. sdm does all that piwiz does (if you choose to use the features)
Pi tools:
Quickly and easily build customized-just-for-you SSDs/SD Cards: https://github.com/gitbls/sdm
Easily run and manage your network's DHCP/DNS servers on a Pi: https://github.com/gitbls/ndm
Easy and secure strongSwan VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

bls
Posts: 2308
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA

Re: SD Card Image Manager

Thu Apr 22, 2021 1:04 am

V4.07 is available and adds the --hotspot switch to quickly and easily confgure a hotspot into your image.

Hotspot networks can be routed, bridged, or loocal only access.
Pi tools:
Quickly and easily build customized-just-for-you SSDs/SD Cards: https://github.com/gitbls/sdm
Easily run and manage your network's DHCP/DNS servers on a Pi: https://github.com/gitbls/ndm
Easy and secure strongSwan VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

bls
Posts: 2308
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA

Re: SD Card Image Manager

Tue Jul 20, 2021 6:29 pm

A few recent updates:
  • --hdmi-ignore-edid added. Sets the correct hdmi_ignore_edid in /boot/config.txt
  • sdm-hotspot can install a hotspot without using sdm at all
  • --lxde-config enables customization of pcmanfm and lxterminal. Your RasPiOS with Desktop can be even more customized!
  • Fixed a few bugs
Pi tools:
Quickly and easily build customized-just-for-you SSDs/SD Cards: https://github.com/gitbls/sdm
Easily run and manage your network's DHCP/DNS servers on a Pi: https://github.com/gitbls/ndm
Easy and secure strongSwan VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

Return to “Advanced users”