cleverca22
Posts: 8638
Joined: Sat Aug 18, 2012 2:33 pm

pi5 suspend to ram

Sat Oct 07, 2023 1:12 am

having looked at the pi5 firmware, i can see that the suspend to ram code is rather simple

all it seems to do, is store the return address (signaled over the mailbox), turn the arm core off, and wait for some event
then write the return addr to ram (i assume the armstub handles it), and turn the arm core back on

this leads to 2 questions:

would it be possible to add multiple wake options?
it seems simple enough to just suspend for say 20 hours, using the VPU timers
configurable gpio edge wake detection, say a PIR motion sensor to resume the system

it doesnt seem that hard to backport this feature to every other pi model, all your really doing is turning the arm power domain off, and then booting it again later

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 3061
Joined: Thu Jul 11, 2013 2:37 pm

Re: pi5 suspend to ram

Sat Oct 07, 2023 3:53 am

In suspend, the entire AP apart from a tiny always-on domain is switched off. There's a simple powerdown state machine controller that can trigger on always-on GPIO pins and resume power to the rest of the AP. No, those pads are not externally accessible - unless the debug UART happens to use them.
cleverca22 wrote:
it doesnt seem that hard to backport this feature to every other pi model, all your really doing is turning the arm power domain off, and then booting it again later
That would require the power domains for the ARM and a retention domain to be separate, which they aren't.
Rockets are loud.
https://astro-pi.org

cleverca22
Posts: 8638
Joined: Sat Aug 18, 2012 2:33 pm

Re: pi5 suspend to ram

Sat Oct 07, 2023 5:42 am

jdb wrote:
Sat Oct 07, 2023 3:53 am
That would require the power domains for the ARM and a retention domain to be separate, which they aren't.
ah, i assumed PM_PROC was all of the ARM stuff
but it can be hard to tell, without the docs

the code looked similar to the wake-on-gpio stuff, but i cant see the trigger for that suspend stuff

User avatar
Gavinmc42
Posts: 8120
Joined: Wed Aug 28, 2013 3:31 am

Re: pi5 suspend to ram

Sat Oct 07, 2023 8:36 am

Initial RP1 doc
https://datasheets.raspberrypi.com/rp1/ ... herals.pdf
Page 5 is a bit about the two M3 micros on the RP1.
Page 12/13 F7 mode, PROC-RIO, not sure if that is Host direct to GPIO or M3 to GPIO, suspect latter.

Page 6, PROC is internal M3's.
Ports prefixed with Proc are accessible only to the management processor complex.
Not sure how yet but indications are the BCM2712 can suspend and the RP1 should be able to wake up from any GPIO.
Further indication, is a M3 can wake up from image sensor light changes.
Since the Ethernet controller is in the RP1, Wake on Lan?

3ma has been a quoted figure, is that including some sort of DRAM refresh?
Or is suspend to RAM only the internal SRAM?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

hippy
Posts: 15868
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: pi5 suspend to ram

Sat Oct 07, 2023 2:14 pm

Gavinmc42 wrote:
Sat Oct 07, 2023 8:36 am
Since the Ethernet controller is in the RP1, Wake on Lan?
https://www.raspberrypi.com/news/introducing-raspberry-pi-5/#comment-1594262 wrote: For wake-on-LAN I think the answer is probably no – the hardware doesn’t make it easy, and we probably won’t get around to fudging it using RP1 software.

cleverca22
Posts: 8638
Joined: Sat Aug 18, 2012 2:33 pm

Re: pi5 suspend to ram

Sat Oct 07, 2023 5:14 pm

Gavinmc42 wrote:
Sat Oct 07, 2023 8:36 am
Page 6, PROC is internal M3's.
i was refering to the videocore PM_PROC
https://github.com/librerpi/rpi-open-fi ... nt.cc#L302
https://github.com/librerpi/rpi-open-fi ... man.h#L185

Code: Select all

#define PM_PROC                                                  HW_REGISTER_RW( 0x7e100110 ) 
   #define PM_PROC_MASK                                          0x007f107f
   #define PM_PROC_WIDTH                                         23
   #define PM_PROC_RESET                                         0000000000
      #define PM_PROC_CFG_BITS                                   22:16
      #define PM_PROC_CFG_SET                                    0x007f0000
      #define PM_PROC_CFG_CLR                                    0xff80ffff
      #define PM_PROC_CFG_MSB                                    22
      #define PM_PROC_CFG_LSB                                    16
      #define PM_PROC_ENAB_BITS                                  12:12
      #define PM_PROC_ENAB_SET                                   0x00001000
      #define PM_PROC_ENAB_CLR                                   0xffffefff
      #define PM_PROC_ENAB_MSB                                   12
      #define PM_PROC_ENAB_LSB                                   12
...
this one is involved in turning on the arm core

cleverca22
Posts: 8638
Joined: Sat Aug 18, 2012 2:33 pm

Re: pi5 suspend to ram

Sun Oct 15, 2023 5:11 am

thinking about it more, the lowest power-state the pi4 has, while still being able to wake itself, is the state that WAKE_ON_GPIO=1 puts it into
where it just reboots, and then pauses before dram init


what is stopping the pi4 firmware, from just turning most of the peripherals/powerdomains off, to enter a similar state (but keeping dram online of course)
and then turning them back on when a gpio wiggles or VPU timer expires?

it wont be as low as the pi5, but it would still be possible
and since the arm is disabled, its still going to be less then just leaving linux idling

User avatar
Gavinmc42
Posts: 8120
Joined: Wed Aug 28, 2013 3:31 am

Re: pi5 suspend to ram

Wed Oct 18, 2023 1:18 am

I would not be surprised if the m3 cores are on the AON domain using a power-gate to keep the leakage low when not in use. It would be very cool to see individual peripheral (GPIO, m3, LAN) suspend enables in the config.txt file, where the enables just control a power-gate. This is not advantageous for a wall-powered setup, but for battery operation it would be much better than any previous RPi board.
I can't wait to get my board in a couple of weeks.
I suspect having that Uart might be handy to monitor power modes.
Turning HDMI off will save power.
There are 8 power sections in that new power chip.
Which GPIO can be used for Uart for RP1 monitoring using a M3?

With a RP1 data sheet at ~1200 pages and no data sheet yet for the Power supply chip things will be interesting for some time.
I assume the DRAM controller is in the BCM2712 but without schematics that is a guess.
Can the RP1 refresh the DRAM?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

cleverca22
Posts: 8638
Joined: Sat Aug 18, 2012 2:33 pm

Re: pi5 suspend to ram

Wed Oct 18, 2023 2:13 am

Gavinmc42 wrote:
Wed Oct 18, 2023 1:18 am
I assume the DRAM controller is in the BCM2712 but without schematics that is a guess.
Can the RP1 refresh the DRAM?
the dram init sequence in the pi5 looks nearly identical to the pi4, so i would assume its the same lpddr4 controller in the 2711 and 2712

the rp1 is bootstrapped over i2c, and that happens after dram init, so the rp1 cant really do anything dram related
phoenix_az wrote:
Tue Oct 17, 2023 10:26 pm
It would be very cool to see individual peripheral (GPIO, m3, LAN) suspend enables in the config.txt file
i can go thru the pi5 firmware, and list off every config.txt it looks for...
  1. disable_l2cache
  2. dtdebug
  3. arm_64bit
  4. enable_gic
  5. arm_peri_high
  6. usb_max_current_enable
  7. dvfs
  8. safe_mode_gpio
  9. hdmi_enable_4kp60
  10. disable_fw_kms_setup
  11. force_eeprom_read
  12. mask_gpu_interrupt0
  13. mask_gpu_interrupt1
  14. max_framebuffers
  15. framebuffer_swap
  16. framebuffer_depth
  17. hdmi_force_hotplug
  18. max_framebuffer_width
  19. max_framebuffer_height
  20. framebuffer_ignore_alpha
  21. init_uart_clock
  22. otg_mode
  23. initial_turbo
  24. force_turbo
  25. disable_commandling_tags
  26. disable_poe_fan
  27. enable_uart
  28. upstream_kernel
  29. camera_auto_detect
  30. display_auto_detect
  31. total_mem
  32. disable_auto_turbo
  33. temp_limit
  34. over_voltage
  35. arm_freq
  36. arm_freq_min
  37. over_voltage_avs
  38. over_voltage_delta
  39. avs_disable
  40. over_voltage_sdram_min
  41. disable_camera_led
  42. dsu_scale
  43. gpu_freq_min
  44. gpu_freq
  45. core_freq
  46. core_freq_min
  47. v3d_freq_min
  48. v3d_freq
  49. h264_freq_min
  50. h264_freq
  51. hevc_freq_min
  52. hevc_freq
  53. isp_freq_min
  54. isp_freq
  55. sdram_freq_min
  56. sdram_freq
  57. remote_gpu_mem
  58. os_prefix
  59. overlay_prefix
  60. kernel
  61. arm_control
  62. ramfsfile
  63. ramfsaddr
  64. auto_initramfs
  65. device_tree_address
  66. device_tree_end
  67. force_core
  68. kernel_address
  69. kernel_old
  70. vpred
  71. avs_temp
  72. desired_osc_freq
  73. desired_osc_freq_boost
  74. temp_soft_limit
  75. sdra,_schmoo
  76. over_voltage_sdram_p
  77. over_voltage_sdram_i
  78. over_voltage_sdram_c
  79. over_voltage_sdram_p_min
  80. over_voltage_sdram_i_min
  81. over_voltage_sdram_c_min
  82. force_pwm_open
  83. audio_pwm_mode
  84. pause_burst_frames
  85. display_default_lcd
  86. display_hdmi_rotate
  87. display_lcd_rotate
  88. ignore_lcd
  89. enable_tvout
  90. avoid_warnings

User avatar
Gavinmc42
Posts: 8120
Joined: Wed Aug 28, 2013 3:31 am

Re: pi5 suspend to ram

Wed Oct 18, 2023 10:14 pm

Most of the mobile DRAM that I have seen have self-refresh.
Good point, time to check DRAM datasheet.

In one vid Eben talked about what the Pi5 does on boot, much less than the old Pi's.
DRAM init is done on boot, for baremetal that is handy, on less thing to worry about, maybe.

Any Linux halt, suspend will be in open source code.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

andrum99
Posts: 2099
Joined: Fri Jul 20, 2012 2:41 pm

Re: pi5 suspend to ram

Wed Oct 18, 2023 10:41 pm

So presumably suspend to RAM will be implemented in Raspberry Pi OS? Nice. Suspend to disk would be handy too: I use that every day on my laptop: close the lid, wait a few seconds and it powers down. When I next need to use it, open the lid and a few seconds later I'm back where I left off - simple. Been doing it like that on Windows for about 23 years - since Windows 2000.

User avatar
Gavinmc42
Posts: 8120
Joined: Wed Aug 28, 2013 3:31 am

Re: pi5 suspend to ram

Wed Oct 18, 2023 10:56 pm

So presumably suspend to RAM will be implemented in Raspberry Pi OS? Nice. Suspend to disk would be handy too:
Thinking about it, it would have to be in the OS because they now have a power button.
Someone just has to reprogram that button to play Tetris :lol:
512KB per-core L2 caches and a 2MB shared L3 cache
Stick a not so tiny RTOS in those cache, on chip so they would be SRAM based?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

timg236
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1482
Joined: Thu Jun 21, 2018 4:30 pm

Re: pi5 suspend to ram

Thu Oct 19, 2023 8:28 am

Suspend to DDR with self-refresh is not supported right now, probably a future firmware enhancement. Once implemented it will switch off VDD_CORE so the ARMs, VPU and caches will be off.

cleverca22
Posts: 8638
Joined: Sat Aug 18, 2012 2:33 pm

Re: pi5 suspend to ram

Thu Oct 19, 2023 9:32 am

timg236 wrote:
Thu Oct 19, 2023 8:28 am
Suspend to DDR with self-refresh is not supported right now, probably a future firmware enhancement. Once implemented it will switch off VDD_CORE so the ARMs, VPU and caches will be off.
ah, that explains why the pi5 firmware looked strange

its got just enough to test all of the psci and mailbox stuff, and "suspend" the arm core for 1 second, but thats it

from what i understand of the hardware, it should be possible to implement the same on the past pi models, but i dont know how much savings youll get when you cant switch VDD_CORE off
the biggest gain youll have, is just that the arm isnt clocked anymore, so pesky background tasks cant eat up battery life

User avatar
Gavinmc42
Posts: 8120
Joined: Wed Aug 28, 2013 3:31 am

Re: pi5 suspend to ram

Thu Oct 19, 2023 10:37 am

Suspend to DDR with self-refresh is not supported right now, probably a future firmware enhancement. Once implemented it will switch off VDD_CORE so the ARMs, VPU and caches will be off.
DDR self refresh would be good.

Not something I thought about before as there was limited power modes on older Pi's but can the Arm core clocks be halted or go into low freq?
Does Arm have separate power than VPUs?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

cleverca22
Posts: 8638
Joined: Sat Aug 18, 2012 2:33 pm

Re: pi5 suspend to ram

Thu Oct 19, 2023 11:30 am

Gavinmc42 wrote:
Thu Oct 19, 2023 10:37 am
Not something I thought about before as there was limited power modes on older Pi's but can the Arm core clocks be halted or go into low freq?
the arm clock could probably be turned off at any time, and then turned back on
Gavinmc42 wrote:
Thu Oct 19, 2023 10:37 am
Does Arm have separate power than VPUs?
there is a PM_PROC power control register, involved in turning the arm core on, when on a pi0-pi3 model
but i dont know how much that actually turns on/off
i dont know how the pi5 has changed things

Return to “General discussion”