{"id":25,"date":"2022-04-14T23:41:12","date_gmt":"2022-04-15T06:41:12","guid":{"rendered":"https:\/\/blogs.oregonstate.edu\/usageoptions\/?p=25"},"modified":"2022-04-21T21:25:26","modified_gmt":"2022-04-22T04:25:26","slug":"linux-kernel-series-memblock","status":"publish","type":"post","link":"https:\/\/blogs.oregonstate.edu\/usageoptions\/2022\/04\/14\/linux-kernel-series-memblock\/","title":{"rendered":"Linux kernel series: memblock"},"content":{"rendered":"\n<p style=\"font-size:1rem\">This post is the first in a series in which I discuss the basics of a component or concept in the Linux kernel. The goal is to explain it in such a way that a beginner C or C++ programmer unfamiliar with the Linux kernel can comprehend it. There will be at least 2 posts in this series, including this one.<\/p>\n\n\n\n<p style=\"font-size:1rem\">This week, the topic is <code>memblock<\/code>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" style=\"font-size:1.75rem\">What is <code style=\"font-size:1.75rem\">memblock<\/code>?<\/h2>\n\n\n\n<p style=\"font-size:1rem\">When programming in <a href=\"https:\/\/en.wikipedia.org\/wiki\/User_space_and_kernel_space\">kernel space<\/a>, you cannot use the C standard library memory allocation functions available in user space, such as <code>malloc<\/code>, <code>free<\/code>, <code>calloc<\/code>, <code>realloc<\/code>, or <code>reallocarray<\/code>.<sup><a href=\"#ref1\">[1]<\/a><\/sup> There are kernel space equivalents, such as <code>kmalloc<\/code> or <code>kzalloc<\/code>, <code>kfree<\/code>, <code>kcalloc<\/code>, <code>krealloc_array<\/code>, <code>krealloc<\/code>,<sup><a href=\"#ref2\">[2]<\/a><\/sup> <a href=\"https:\/\/www.kernel.org\/doc\/html\/latest\/core-api\/memory-allocation.html\">and more<\/a>. However, these memory allocators are set up during the booting process. In the period of time before these allocators are available, the system still needs to allocate memory. During that period of time, a specialized allocator called <code>memblock<\/code> is used to allocate memory.<sup><a href=\"#ref3\">[3]<\/a><\/sup><\/p>\n\n\n\n<p style=\"font-size:1rem\"><code>memblock<\/code> views the system memory as a collection of contiguous regions. <code>memblock<\/code> is made up of memblock types, which are themselves made up of multiple memory regions. There are several structures and functions for working with <code>memblock<\/code>.<sup><a href=\"#ref4\">[4]<\/a><\/sup><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" style=\"font-size:1.75rem\">Structures<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\" style=\"font-size:1.25rem;text-transform:lowercase\">struct memblock<\/h3>\n\n\n\n<p style=\"font-size:1rem\">The <code>struct memblock<\/code> wraps the metadata for the <strong>memory<\/strong> and <strong>reserved<\/strong> memblock types, along with <code>current_limit<\/code>, the physical address of the current allocation limit, and <code>bottom_up<\/code>, the allocation direction. The allocation limit limits allocations to memory that is currently accessible and can be set with <code>memblock_set_current_limit(phys_addr_t limit)<\/code>. The <code>memblock<\/code> structure is statically initialized at build time.<\/p>\n\n\n\n<pre class=\"wp-block-code has-light-gray-background-color has-background\" style=\"font-size:1rem\"><code><sup><a href=\"#ref5\">&#091;5]<\/a><\/sup>\nstruct memblock {\n\tbool bottom_up;\n\tphys_addr_t current_limit;\n\tstruct memblock_type memory;\n\tstruct memblock_type reserved;\n};<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" style=\"font-size:1.25rem;text-transform:lowercase\">struct memblock_type<\/h3>\n\n\n\n<p style=\"font-size:1rem\">An array of several memory regions of the same type can be represented with <code>struct memblock_type<\/code>. This struct defines an array of memory regions <code>regions<\/code> of type <code>name<\/code> with <code>cnt<\/code> number of regions. The total size of the array is <code>max<\/code>, and the size of all the regions is <code>total_size<\/code>.<\/p>\n\n\n\n<pre class=\"wp-block-code has-light-gray-background-color has-background\" style=\"font-size:1rem\"><code><sup><a href=\"#ref5\">&#091;5]<\/a><\/sup>\nstruct memblock_type {\n\tunsigned long cnt;\n\tunsigned long max;\n\tphys_addr_t total_size;\n\tstruct memblock_region *regions;\n\tchar *name;\n};<\/code><\/pre>\n\n\n\n<p style=\"font-size:1rem\">The memory region types are <strong>memory<\/strong>, <strong>reserved<\/strong>, and <strong>physmem<\/strong>. <\/p>\n\n\n\n<ul class=\"wp-block-list\" style=\"font-size:1rem\"><li><strong>memory<\/strong> is the physical memory available to the kernel.<\/li><li><strong>reserved<\/strong> refers to the regions that were actually allocated.<\/li><li><strong>physmem<\/strong> is the total physical memory available during boot regardless of whether the kernel can access it. It is not available on some architectures.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" style=\"font-size:1.25rem;text-transform:lowercase\">struct memblock_region<\/h3>\n\n\n\n<p style=\"font-size:1rem\">The <code>struct memblock_region<\/code> represents a memory region with base address <code>base<\/code>, size <code>size<\/code>, memory region attributes <code>flags<\/code>, and NUMA node id <code>nid<\/code>. NUMA is a system with <a href=\"https:\/\/www.kernel.org\/doc\/html\/latest\/vm\/numa.html\">non-uniform memory access<\/a>. <\/p>\n\n\n\n<pre class=\"wp-block-code has-light-gray-background-color has-background\" style=\"font-size:1rem\"><code><sup><a href=\"#ref5\">&#091;5]<\/a><\/sup>\nstruct memblock_region {\n\tphys_addr_t base;\n\tphys_addr_t size;\n\tenum memblock_flags flags;\n#ifdef CONFIG_NUMA\n\tint nid;\n#endif\n};<\/code><\/pre>\n\n\n\n<p style=\"font-size:1rem\">The memory region attributes are defined in <code>enum memblock_flags<\/code>.<\/p>\n\n\n\n<pre class=\"wp-block-code has-light-gray-background-color has-background\" style=\"font-size:1rem\"><code><sup><a href=\"#ref5\">&#091;5]<\/a><\/sup>\nenum memblock_flags {\n\tMEMBLOCK_NONE\t\t= 0x0,\t\/* No special request *\/\n\tMEMBLOCK_HOTPLUG\t= 0x1,\t\/* hotpluggable region *\/\n\tMEMBLOCK_MIRROR\t\t= 0x2,\t\/* mirrored region *\/\n\tMEMBLOCK_NOMAP\t\t= 0x4,\t\/* don't add to kernel direct mapping *\/\n\tMEMBLOCK_DRIVER_MANAGED = 0x8,\t\/* always detected via a driver *\/\n};<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\" style=\"font-size:1.75rem\">Functions<\/h2>\n\n\n\n<p style=\"font-size:1rem\"><code>memblock<\/code> is initialized in <code>setup_arch<\/code>() and taken down in <code>mem_init<\/code>().<\/p>\n\n\n\n<p style=\"font-size:1rem\"><code>memblock<\/code> has many useful APIs. These APIs include functions for adding and removing memory regions, and allocating and freeing memory, among <a href=\"https:\/\/www.kernel.org\/doc\/html\/latest\/core-api\/boot-time-mm.html#c.for_each_physmem_range\">many other functions<\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" style=\"font-size:1.25rem;text-transform:capitalize\">Adding and removing<\/h3>\n\n\n\n<p style=\"font-size:1rem\">These functions add or remove a memblock region with the given base address and size. They return an int indicating whether the operation succeeded. The <code>memblock_add_node()<\/code> version adds the region within a NUMA node and includes parameters for the node id and memblock flags.<\/p>\n\n\n\n<pre class=\"wp-block-code has-light-gray-background-color has-background\" style=\"font-size:1rem\"><code><code>memblock_add(base, size)<\/code>memblock_add_node(base, size, nid, flags)\nmemblock_remove(base, size)<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" style=\"font-size:1.25rem;text-transform:capitalize\">Allocating<\/h3>\n\n\n\n<p style=\"font-size:1rem\">These functions allocate a memory block and return the physical address of the memory.<\/p>\n\n\n\n<pre class=\"wp-block-code has-light-gray-background-color has-background\" style=\"font-size:1rem\"><code>memblock_phys_alloc(size, align)\nmemblock_phys_alloc_range(size, align, start, end)\nmemblock_phys_alloc_try_nid(size, align, nid)<\/code><\/pre>\n\n\n\n<p style=\"font-size:1rem\">These functions allocate a memory block and return the virtual address of the memory.<\/p>\n\n\n\n<pre class=\"wp-block-code has-light-gray-background-color has-background\" style=\"font-size:1rem\"><code>memblock_alloc(size, align)\nmemblock_alloc_from(size, align, min_addr)\nmemblock_alloc_low(size, align)\nmemblock_alloc_node(size, align, nid)<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" style=\"font-size:1.25rem;text-transform:capitalize\">Freeing<\/h3>\n\n\n\n<p style=\"font-size:1rem\">The functions <code>memblock_free()<\/code> and <code>memblock_phys_free()<\/code> free a boot memory block that was previously allocated; it is not released to the buddy allocator (which handles allocation after boot process completes). The <code>*ptr<\/code> parameter in <code>memblock_free()<\/code> is the starting address of the boot memory block. The functions <code>memblock_free_all()<\/code> and <code>memblock_free_late()<\/code> free pages to the buddy allocator.<\/p>\n\n\n\n<pre class=\"wp-block-code has-light-gray-background-color has-background\" style=\"font-size:1rem\"><code>memblock_free(*ptr, size)\nmemblock_phys_free(base, size)\nmemblock_free_all(void)\nmemblock_free_late(base, size)<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\" style=\"font-size:1.75rem\">References<\/h2>\n\n\n\n<ol class=\"wp-block-list\" style=\"font-size:1rem\"><li id=\"ref1\"><a href=\"https:\/\/man7.org\/linux\/man-pages\/man3\/free.3.html\">malloc(3) \u2014 Linux manual page<\/a><\/li><li id=\"ref2\"><a href=\"https:\/\/www.kernel.org\/doc\/html\/latest\/core-api\/mm-api.html#c.kmalloc\">Memory Management APIs \u2014 The Linux Kernel documentation<\/a><\/li><li id=\"ref3\"><a href=\"https:\/\/www.kernel.org\/doc\/html\/latest\/core-api\/boot-time-mm.html#\">Boot time memory management \u2014 The Linux Kernel documentation<\/a><\/li><li id=\"ref4\"><a href=\"https:\/\/www.kernel.org\/doc\/html\/latest\/core-api\/boot-time-mm.html#functions-and-structures\">Boot time memory management \u2014 The Linux Kernel documentation: Functions and structures<\/a><\/li><li id=\"ref5\">Code snippet from <a href=\"https:\/\/github.com\/torvalds\/linux\/blob\/master\/include\/linux\/memblock.h\">linux\/memblock.h<\/a><\/li><\/ol>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>This post is the first in a series in which I discuss the basics of a component or concept in the Linux kernel. The goal is to explain it in such a way that a beginner C or C++ programmer &hellip; <a href=\"https:\/\/blogs.oregonstate.edu\/usageoptions\/2022\/04\/14\/linux-kernel-series-memblock\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":12377,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[17,3,16,4],"class_list":["post-25","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-c","tag-linux","tag-memblock","tag-outreachy"],"_links":{"self":[{"href":"https:\/\/blogs.oregonstate.edu\/usageoptions\/wp-json\/wp\/v2\/posts\/25","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.oregonstate.edu\/usageoptions\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.oregonstate.edu\/usageoptions\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/usageoptions\/wp-json\/wp\/v2\/users\/12377"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/usageoptions\/wp-json\/wp\/v2\/comments?post=25"}],"version-history":[{"count":21,"href":"https:\/\/blogs.oregonstate.edu\/usageoptions\/wp-json\/wp\/v2\/posts\/25\/revisions"}],"predecessor-version":[{"id":64,"href":"https:\/\/blogs.oregonstate.edu\/usageoptions\/wp-json\/wp\/v2\/posts\/25\/revisions\/64"}],"wp:attachment":[{"href":"https:\/\/blogs.oregonstate.edu\/usageoptions\/wp-json\/wp\/v2\/media?parent=25"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/usageoptions\/wp-json\/wp\/v2\/categories?post=25"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.oregonstate.edu\/usageoptions\/wp-json\/wp\/v2\/tags?post=25"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}