In the final session in the memory-management track of the 2024 Linux Storage,
Filesystem, Memory-Management and BPF Summit, the exhausted group of
developers looked one more time at the use of huge pages and the associated
problem of memory fragmentation. At its worst, this problem can make huge
pages harder (and more expensive) to allocate. Luis Chamberlain, who ran
the session, felt that people were worried about this problem, but that
there was little data on how severe it truly is.
A longstanding tradition in the memory-management track of the Linux Storage,
Filesystem, Memory-Management and BPF Summit is a session with
maintainer Andrew Morton to discuss the overall state of the community and
the development process. The 2024 gathering upheld that tradition toward
the end of the final day of the event. It seems that Morton and the
assembled developers were all happy with how memory-management work is
going, but there is always room for improvement.
Security updates have been issued by Debian (less), Mageia (chromium-browser-stable), SUSE (apache2, java-1_8_0-openj9, kernel, libqt5-qtnetworkauth, and openssl-3), and Ubuntu (netatalk and python-cryptography).
Geoff Huston suggests
that it is time to give up on DNSSEC and look for a better way to secure
the Internet namespace.
What appears to be very clear (to me at any rate!) is that DNSSEC
as we know it today is just not going anywhere. It’s too complex,
too fragile and just too slow to use for the majority of services
and their users. Some value its benefits highly enough that they
are prepared to live with its shortcomings, but that’s not the case
for the overall majority of name holders and for the majority of
users, and no amount of passionate exhortations about DNSSEC will
change this.
One of the long-term goals of the folio conversion in the kernel’s
memory-management subsystem is the replacement of the page
structure, which describes a page of physical memory, with an eight-byte
“memory descriptor”. This change would reduce the overhead of tracking
physical memory, increase type safety, and make memory management more
flexible. Thus far, though, details on what the memory-descriptor future
will look like have been relatively scarce. At the 2024 Linux Storage,
Filesystem, Memory-Management and BPF Summit, Matthew Wilcox led a
discussion to try to fill in the picture somewhat.
Linus Torvalds released 6.10-rc1 and closed
the 6.10 merge window on May 26. By that time, 11,534 non-merge
changesets had been pulled into the mainline for the next release; nearly
5,000 of those came in after “The first half of
the 6.10 merge window” was written. While the latter half of the merge
window tends to focus more on fixes, there was also a lot of new
functionality that landed during this time.
The maple tree data structure was added
during the 6.1 development cycle; since then, it has taken its
place at the core of the kernel’s memory-management subsystem.
Unsurprisingly, work on maple trees is not yet done. Maple-tree maintainer
Liam Howlett ran a session in the memory-management track of the 2024 Linux Storage,
Filesystem, Memory-Management and BPF Summit to discuss the current
state of the maple tree and which features can be expected next.
Linus has released
6.10-rc1 and closed the merge window for this release. For reasons
that have not been spelled out, the codename for the release has been
changed to “Baby Opossum Posse”.
Using huge pages has been known for years to improve the performance of
many workloads. But traditional huge pages, often sized by the CPU at 2MB,
can be difficult to allocate and can waste memory due to internal
fragmentation. Driven by both the folio transition and hardware
improvements, attention to smaller, multi-size transparent huge pages
(mTHPs) has been on the rise. In two memory-management-track sessions at
the 2024 Linux Storage,
Filesystem, Memory-Management and BPF Summit, developers discussed the
kernel’s ability to reliably allocate mTHPs and the performance gains that
result.
The original Linux kernel, posted in 1991, ran on a system with a 4KB page
size. Over 30 years later, most of us are still running on systems
with 4KB pages, even though the amount of installed memory has grown by a
few orders of magnitude. It is generally accepted that using large page
sizes results in better performance for most applications, but allocating
larger pages is often difficult. During a memory-management session at the 2024 Linux Storage,
Filesystem, Memory-Management and BPF Summit, Yu Zhao presented his
ideas on improving the allocation of huge pages in the kernel.
The kernel contains a pair of related filesystems that, among other things,
can be used for shared-memory applications; shmem is an internal mechanism
used within the kernel, while the tmpfs filesystem is mounted and
accessible from user space. As is the case elsewhere in the kernel, these
subsystems would benefit from the addition of large-folio support. During
a joint storage, filesystem, and memory-management session at the 2024 Linux Storage,
Filesystem, Memory-Management and BPF Summit, Daniel Gomez talked about
the work he is doing to add that support.
This
Graphite blog post retells the history of the BitKeeper fiasco and the
dawn of the Git era.
When we think of history, we often romanticize it as being born of
a sudden stroke of inspiration. But the creation of git shows the
far harsher reality of invention: a slowly escalating disagreement
over a license; the need for a scrappy backup solution to unblock
work; and then continued polishing and iteration through years and
years, led not by the inventor, but rather a community.
For those who weren’t around in those days, a perusal of the LWN coverage
from the time might be of interest too, including:
Swapping may be a memory-management technique at its core, but its
implementation also involves the kernel’s filesystem and storage layers.
So it is not surprising that a session on the kernel’s swap abstraction
layer, led by Chris Li at the 2024 Linux Storage,
Filesystem, Memory-Management and BPF Summit, was held jointly by all
three of those tracks. Li has some ambitious ideas for an improved
subsystem, but getting to a workable implementation may not be easy.
Almost immediately after the merging of control
groups, kernel developers set their sights on reimplementing them
properly. The second version of the control-group API started trickling
into the kernel around the 3.16 release in 2014 and users have long since
been encouraged to migrate, but support for (and users of) the initial API
remain. At the 2024
Linux Storage, Filesystem, Memory-Management and BPF Summit,
memory-management developers discussed whether (and when) it might be
possible to remove the version-1 memory controller. The session was led by
Shakeel Butt and (participating remotely) Roman Gushchin.
The term “memory model” is used in a couple of ways within the kernel.
Perhaps the more obscure meaning is the memory-management subsystem’s view
of how physical memory is organized on a given system. A proper
representation of physical memory will be more efficient in terms of memory
and CPU use. Since hardware comes in numerous variations, the kernel
supports a number of memory models to match; see this article for details. At the 2024 Linux Storage,
Filesystem, Memory-Management and BPF Summit, Oscar Salvador,
presenting remotely, made the case for removing one of those models.
For every page of memory in the system, the kernel maintains a set of page
flags describing how the page is used and various aspects of its current
state. Space for page flags has been in chronic short supply, leading to a desire to
eliminate or consolidate them whenever possible. That objective, though,
is hampered by the fact that the purpose of many page flags is not well
understood. In a memory-management-track session at the 2024 Linux Storage,
Filesystem, Memory-Management and BPF Summit, Matthew Wilcox set out to
cooperatively update the page-flag documentation to improve that situation.
The problem of sharing page tables across processes has been discussed
numerous times over the years, Khaled Aziz said at the beginning of his 2024 Linux Storage,
Filesystem, Memory-Management and BPF Summit session on the topic. He
was there to, once again, talk about the proposed mshare() system call (which, in its
current form, is no longer actually a system call but the feature still
goes by that name) and to see what can be done to finally get it into the
mainline.
The kernel’s hugetlbfs
subsystem was the first mechanism by which the kernel made huge pages
available to user space; it was added to the 2.5.46 development kernel in
2002. While hugetlbfs remains useful, it is also viewed as a sort of
second memory-management subsystem that would be best unified with the rest
of the kernel. At the 2024 Linux Storage,
Filesystem, Memory-Management and BPF Summit, Peter Xu raised the
question of what that unification would involve and what the first steps
might be.
The collective thoughts of the interwebz
Manage Consent
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.