According to a DOJ press release, the FBI was able to delete the Chinese-used PlugX malware from “approximately 4,258 U.S.-based computers and networks.”
To retrieve information from and send commands to the hacked machines, the malware connects to a command-and-control server that is operated by the hacking group. According to the FBI, at least 45,000 IP addresses in the US had back-and-forths with the command-and-control server since September 2023.
It was that very server that allowed the FBI to finally kill this pesky bit of malicious software. First, they tapped the know-how of French intelligence agencies, which had recently discovered a technique for getting PlugX to self-destruct. Then, the FBI gained access to the hackers’ command-and-control server and used it to request all the IP addresses of machines that were actively infected by PlugX. Then it sent a command via the server that causes PlugX to delete itself from its victims’ computers.
превод от френски Владимир Сунгарски, София: изд. „Парадокс“, 2020
В самото начало на тази книга се чудех само толкова ли е достатъчно, за да вземе „Гонкур“. В края ѝ вече съм напълно наясно защо „А след тях и децата им“ на Никола Матийо е получила това най-високо френско литературно отличие през 2018 г. Между началото и края на едно десетилетие, между две социално-икономически реалности, между две огледални кражби, затворили сюжетната рамка,
Матийо сътворява един проникновен портрет на френското общество от 90-те години, подобно на някогашните френски майстори на реализма.
Неслучайно писателят споделя в интервюта, че е чел доста Ани Ерно (тук може да прочетете и рецензията за нейния роман „Годините“, донесъл ѝ Нобелова награда), както и Зола, Флобер, Селин, Колет, Жионо, Стайнбек и др. И макар да описва всички безжалостно настъпващи промени под валяка на историята, посланието на този роман в крайна сметка е, че може би нищо не се променя във въртележката от поколения, както пределно ясно ни посочва и заглавието. Бунтът и приемствеността, мечтите и разочарованията, стремежът и конформизмът са обречени да се редуват и поглъщат едни други.
Всички ставаме родителите си, дори и обстоятелствата привидно да се менят, а светът непрестанно да се обновява.
„А след тях и децата им“ (сдобил се и с филмова адаптация тази година) е хронологичен билдунгсроман, прекъсван инцидентно от кратки реминисценции. В неговата орбита са редица герои – младежи и техните родители, но фокусът пада върху трима тийнейджъри, родени с малка разлика през 70-те: Антони – френско момче от работническо семейство, малко по-големият син на марокански имигранти Хасин, както и Стефани, която принадлежи към по-заможната средна класа. Частите на книгата са разделени между годините (следващи през две), когато се развива действието – от 1992-ра до 1998-ма.
Матийо е написал роман, който благодатно би могъл да се чете през марксическата литературна теория, тъй като описва в дълбочина класово (и етнически) разделеното френско общество и извежда личната история като функция и проявление на обществено-икономическата.
Той поставя на изпитание смисъла на „свободата, братството и равенството“ на фона на „кризата, мизерията, социалната катастрофа“.
Залагайки на социалния детерминизъм, писателят ни води към неизбежния извод, че почти всичко е предопределено от средата. Че между „обстоятелства, личния мързел и общото потисничество“ причините за провала са смътни. Че в крайна сметка всеки порив и амбициозен опит за надмогване на даденостите дерайлира.
Семействата на Антони и Хасин са нефункционални и работнически (а на Хасин и имигрантско), това на Стефани – доста по-предприемчиво и заможно, но затънало в своята повърхностна еснафщина. Младежите живеят в сякаш екзистенциално отдалечено от Париж провинциално градче и дори не са ходили в столицата. До неотдавна родното им място е било средище на металургичната промишленост в района, но сега, поради процесите на деиндустриализация, останките на бившия завод, „веднъж зарязани и продадени за скрап“, са оставили „жестоки кървави дупки в града“. Долината е западнала, а мъжете, останали без традиционния си поминък, трябва да се оправят с помощи, обезщетения или поредица от недостатъчно платени работи.
Матийо майсторски рисува поколението на родителите – отломка от онзи католически, традиционен и патриархален поствоенен свят, възпитавано в ценността на общия труд, на това да блъскаш заедно с другите. То е неадекватно на фона на постепенното преориентиране на икономиката; на убиването на малките бизнеси за сметка на безличните търговски мастодонти; на нарастващия индивидуализъм; на корпоративната офис култура и въвеждането на технологиите; на нуждата от нови социални умения, нагласи и кълнене в клишета. Имиграционните потоци променят посоките си и сега самите местни търсят възможности в Германия и Люксембург. Чрез символичната смърт на един от синдикалните лидери, популярна фигура от работническите протести, романът ознаменува този преход. Краят на работници и синдикати е описан така:
Стогодишната солидарност се разтваряше в голямата вана от конкуриращи се сили. Навсякъде нови, малки, неблагодарни, зле платени работи, пълни с унизително раболепничество, заместваха някогашното колективно размазване от бачкане. Производството вече нямаше смисъл. Използваха се думи като връзки с обществеността, качество на услугата, комуникационна стратегия, удовлетворени клиенти. Всичко беше станало малко, изолирано, мъгляво, педерастко по дух.
В новата „икономика, призвана да менажира мизерията“ и неугледния, едва закърпен бит, имаме всички маркери на живота тип lowlife и redneck: нефункционални домове, изплащани с десетилетия; хроничен недостиг на пари; треперене за сметките и за несигурната, механична работа; нездравословен начин на живот; липса на емоционална интелигентност в отношенията; промиване на мозъците от масовата тв култура и т.н. Матийо говори за „конюнктурните нещастия“ в средата на героя си: изоставени, разведени, алкохолизирани, пребити по пътищата, пукясали рано, уволнени, съсипани, инвалидизирани, болни, прибягващи редовно до ксанакс или дрога хора.
Най-големите късметлии бяха успели да си купят къща с дворче, плод на двайсет години саможертви. Тези порутени сгради бяха живото доказателство за провала на един свят и неговите архитекти.
И въпреки всичко това изглежда някак нормално, недраматично, естествено. Самите те дребни дилъри и хулигани в началото, без каквито и да било културни интереси освен обичайните футбол, чукане, пафкане, електронни игри, мотори, киснене с приятели, с абстрактни и неясни мечти и идея за свобода, опираща най-вече до достатъчното пари, Антони и Хасин са по-скоро тъжни, отколкото катарзисно трагични образи. Хасин ще поиска да бъде хитър и вездесъщ цар на дрогата от Мароко към Европа, но ще пострада от собствената си наивност и от упойващото ежедневие в интимна двойка (която е едновременно капан за индивидуалността, но и единственият начин да посрещнеш живота поне с някакъв съюзник). Антони завистливо ще мечтае за живота на своята дългогодишна тръпка Стефани, както си го представя:
Свят на летни вили, семейни снимки, разтворена книга върху шезлонга, голямо куче под черешовото дърво, онова безоблачно щастие, което виждаш в списанията пред зъболекарския кабинет.
Преследвайки невъзможния призрак на любовта, той ще го (анти)сублимира просто до сексуалното желание. Но посланието на романа ще бъде препотвърдено от невъзможността двамата да се слеят еротично докрай, до удовлетворителна кулминация. Този почти вулгарен в наивността и натурализма си копнеж ще отрази социалния, материалния, екзистенциалния такъв – оставайки си неудовлетворен, прекъсван, провалян…
Преходът между класите е неосъществим. Изходът – невъзможен. В крайна сметка и Антони (чрез записването в армията), и Хасин (чрез намирането на работа във фирма, която събаря стари и ненужни сгради) пренасочват фрустрацията, агресията, нуждата да разрушат наследството си, „непрестанното желание [Антони] да удря, да си причинява болка, да се блъска право в стените“.
Тежка, задушаваща, обездвижваща жега цари във всички части на романа – действието се развива винаги през летните месеци и ваканция, когато и работа, и училище спират. Жегата се превръща във всеобхватна метафора на тази съдбоносна житейска застоялост, парализа, безизходност. И двамата младежи търсят тръпката, раздвижването в изживяванията на ръба, в опасността, в предизвикването на съдбата – чрез скоростта на моторите или насилието.
Страхуваше се, беше прелестно. (Антони)
Понякога смъртта му се струваше нещо примамливо. (Хасин)
Но те не могат да осъществят докрай дори насилието, когато периодично биват изправени един срещу друг уж като врагове (неслучайно повечето пъти това се случва, докато уринират, една от най-уязвимите за мъжа ситуации). И въпреки че допирните им точки не значат нищо за тях, Матийо ще събере двамата символично върху един мотор, за да ни покаже еднаквостта им. Единият – с баща възпламеним, склонен към бурни реакции и насилие, зависим от алкохола, другият – с баща примирен, унизен, драпащ десетилетия да свърже двата края на имигрантското си битие. Но и Антони, и Хасин – счупени, макар и всеки посвоему.
В тази връзка Матийо изгражда много убедителни, макар и малко крайни тийнейджърски образи спрямо времето – книгата изобилства от познати и у нас реалии, жалони, музика, филми, заглавия, плакати, марки. Главите са наречени на базови парчета от онова време. Например сцената, озвучена от Smells Like Teen Spirit, в която всички купонястващи се клатят в един ритъм в своите карирани ризи, е прекрасно обобщение на поколението:
Във всеки град, роден от този деиндустриализиран и недвусмислен свят, във всяко западнало място хлапетии, останали без мечти, слушаха тази сиатълска група, която се казваше „Нирвана“. Пускаха си дълги коси и се опитваха да превърнат тъгата си в гняв, депресията си – в децибели. Раят беше безвъзвратно изгубен, революцията нямаше да се случи; оставаше единствено да се вдига шум.
Езикът на Матийо също е директен, грубоват, циничен, недодялан, пълен с жаргонни думи (малко странно или може би твърде локално подбрани на български от преводача). Но тук и там той подпъхва красиви, поетични авторови фрази, които по странно красив начин засилват усещането за предопределеност, за фатална безизходност, за всеобща тъга на това поколение, което е все още ошамарвано вместо разбирано от родителите си. Изобилстват и натуралистичните сексуални сцени. Дори романтиката се случва насред тинята или разрухата, на места, които съвсем не я предполагат. Както е на тази възраст, както е в този живот, мръсното се смесва с чистото, вулгарното с невинното, ниското с високото.
Не на последно място в романа – разбираемо – изобилства расизмът от страна на белия провинциален французин от работническата класа, който обаче е горе-долу на същия хал. Подредбата е ясна: главестите, имигрантите работници от Източна и Южна Европа, арабеските, резняците, чернилките и т.н. Всички, които могат да поемат вината. Този въпрос не е решен от Матийо мелодраматично или дидактично, а съвсем честно. Защото макар да принадлежат на различни етноси и обкръжения, световете на Антони и Хасин са еднакво изпълнени с безперспективност. Счупването между поколенията (дори срамът на Хасин от дълголетния акцент и лош френски на баща му) не е достатъчно, за да се изтръгнеш. Не е достатъчно желанието да успее по друг, дързък, цялостен начин, който да го издигне много над жалкото съществуване на баща му, чието относителното задържане на повърхността се дължи на „професионалните оскърбления, на унизителния труд, на изолацията, на самата дума имигрант“, която се използва навсякъде и ги поставя под общ знаменател.
Като възможно обединение обаче Матийо предлага единствено илюзията на футбола (успеха на Франция на Световното през 1998-ма) и образа на Зинедин Зидан. Това единение е почти сексуално и фанатично по природа, но е крехко, каквато е несигурността и преходността на всяка победа. В нея само временно изчезват разликите, дълговете, историята.
Написан реалистично, романът не предлага оптимистична теория за „класово израстване“. Грешките, провалите, слабостите като че ли тегнат наследствено. И тъй като в тази книга спойлерите определено имат значение, понеже цялостната характеризация на героите минава предимно през поведението и диалозите им, нека кажем само, че дори амбициралата се Стефани успява само донякъде. През нейния образ авторът разкрива познанията си и за дебрите на френската образователна система с нейните строги йерархии и производство на кадри – в мнозинството си те са от „голямата и язвителна категория хора, които са свръхквалифицирани и неизползвани“, които разбират всичко, но не могат нищо. В крайна сметка тя, която презира угодливостта, посредствеността и дребнавостта на „нормалните си родители“, без да си дава сметка за цената на тази нормалност, може би също някой ден ще превърне в част от
тези жени, които поколение след поколение се сриваха, превръщаха се наполовина в слугини, чиято единствена задача бе да се грижат за оцеляването на едно потомство, обречено да изпита същите радости и скърби.
Финалната сцена в романа символично събира Антони с майка му на пикник край езерото, около което кипи социалният живот в провинциалното им градче. Близо до тях (както някога и те двамата са правели в детството и младостта си) се забавляват нови тийнове. В тази последна картина три поколения остават свързани невидимо от нишките на гнева и меланхолията. И от тегнещия над всеки по реда си въпрос: „Къде е животът, бе, деба?“ Именно с този прочит на „свободата, братството и равенството“ ни оставя Никола Матийо.
Активните дарители на „Тоест“ получават постоянна отстъпка в размер нa 20% от коричната цена на всички заглавия от каталога на издателство „Парадокс“, както и на няколко други български издателства в рамките на партньорската програма Читателски клуб „Тоест“. За повече информация прочетете на toest.bg/club.
Никой от нас не чете единствено най-новите книги. Тогава защо само за тях се пише? „На второ четене“ е рубрика, в която отваряме списъците с книги, публикувани преди поне година, четем ги и препоръчваме любимите си от тях. Рубриката е част от партньорската програма Читателски клуб „Тоест“. Изборът на заглавия обаче е единствено на авторите – Стефан Иванов и Антония Апостолова, които биха ви препоръчали тези книги и ако имаше как веднъж на две седмици да се разходите с тях в книжарницата.
Briefs: rsync vulnerabilities; Linux Mint 22.1; Git v2.48.9; Libvirt v11.0.0; Rust 1.84.0; RIP Helen Borrie, Paolo Mantegazza, and Bill Gianopoulos; SFC lawsuit; Quotes; …
Announcements: Newsletters, conferences, security updates, patches, and more.
Working with customers, our security teams detected an increase of data encryption events in S3 that used an encryption method known as server-side encryption using client-provided keys (SSE-C). While this is a feature used by many customers, we detected a pattern where a large number of S3 CopyObject operations using SSE-C began to overwrite objects, which has the effect of re-encrypting customer data with a new encryption key. Our analysis uncovered that this was being done by malicious actors who had obtained valid customer credentials, and were using them to re-encrypt objects.
It’s important to note that these actions do not take advantage of a vulnerability within an AWS service—but rather require valid credentials that an unauthorized user uses in an unintended way. Although these actions occur in the customer domain of the shared responsibility model, AWS recommends steps that customers can use to prevent or reduce the impact of such activity.
Using active defense tools, we have implemented automatic mitigations that will help to prevent this type of unauthorized activity in many cases. These mitigations have already prevented a high percentage of attempts from succeeding, without customers taking steps to protect themselves. However, the threat actors used valid credentials, and it is difficult for AWS to reliably distinguish valid usage from malicious use. Therefore, we recommend that customers follow best practices to mitigate risk.
We recommend that customers implement these four security best practices to protect against the unauthorized use of SSE-C:
Block the use of SSE-C unless required by an application
Implement data recovery procedures
Monitor AWS resources for unexpected access patterns
Implement short-term credentials
I. Block the use of SSE-C encryption
If your applications don’t use SSE-C as an encryption method, you can block the use of SSE-C with a resource policy applied to an S3 bucket, or by a resource control policy (RCP) applied to an organization in AWS Organizations.
Resource policies for S3 buckets are commonly referred to as bucket policies and allow customers to specify permissions for individual buckets in S3. A bucket policy can be applied using the S3 PutBucketPolicy API operation, the AWS Command Line Interface (CLI), or through the AWS Management Console. Learn more about how bucket policies work in the S3 documentation. The following example shows a bucket policy that blocks SSE-C request for a bucket called your-bucket-name>.
RCPs allow customers to specify the maximum available permissions that apply to resources across an entire organization in AWS Organizations. An RCP can be applied by using the AWS Organizations UpdatePolicy API operation, the AWS Command Line Interface (CLI), or through the AWS Management Console. Learn more about how RCPs work in the AWS Organizations documentation. The following example shows an RCP that blocks SSE-C requests for buckets in the organization.
Without data protection mechanisms in place, data recovery times can be longer. As a data protection best practice, we recommend that you protect against data being overwritten and that you maintain a second copy of critical data.
Enable S3 Versioning to keep multiple versions of an object in a bucket, so that you can restore objects that are accidentally deleted or overwritten. It is important to note that versioning may increase storage costs, especially for applications that frequently overwrite objects in a bucket. In this case, consider implementing S3 Lifecycle policies to manage older versions and control storage costs.
Additionally, copy or take backups of critical data to a different bucket and perhaps to a different AWS account or AWS Region. To do this, you can use S3 replication to automatically copy objects between buckets. These buckets can reside in the same or in different AWS accounts, as well as in the same or in different AWS Regions. S3 replication also offers an SLA for customers that have more stringent RPO (Recovery Point Objective) and RTO (Recovery Time Objective) requirements. Alternatively, you can use AWS Backup for S3, which is a managed service that automates periodic backup of S3 buckets.
III. Monitor AWS resources for unexpected access patterns
Without monitoring, unauthorized actions on S3 buckets may go unnoticed. We recommend that you use tools such as AWS CloudTrail or S3 server access logs to monitor access to your data.
You can use AWS CloudTrail to log events across AWS services (including Amazon S3) and even combine logs into a single account to make them available to your security teams to access and monitor. You can also create CloudWatch alarms based on specific S3 metrics or logs to alert on unusual activity. These alerts can help you identify anomalous behavior quickly. You can also set up automation that uses Amazon EventBridge and AWS Lambda to automatically take corrective measures. See this topic in the S3 documentation for an example implementation of a setup used to scan all buckets across an organization and apply S3 Block Public Access.
IV. Implement short-term credentials
The most effective approach to mitigating the risk of compromised credentials is to never create long-term credentials in the first place. Credentials that do not exist cannot be exposed or stolen, and AWS provides a rich set of capabilities that alleviate the need to ever store credentials in source code or in configuration files.
All of these technologies rely on the AWS Security Token Service (AWS STS) to issue temporary security credentials that can control access to AWS resources without distributing or embedding long-term AWS security credentials within an application, whether in code or in configuration files.
Summary
Detecting unintended encryption techniques like this in your environment requires vigilance and support. In this post, we highlighted the most common indicators to look for. As your security teams work to constantly protect your environment, know that a number of teams at AWS—including the AWS Customer Incident Response Team (CIRT), Amazon Threat Intelligence, and services teams like the Amazon S3 team—are working diligently to innovate, collaborate, and share insights to help protect your valuable data.
In this post, we provided an update on this recent threat to customer data and highlighted four security best practices that customers can use to protect against the risk of bad actors using SSE-C to encrypt data by using lost or stolen AWS credentials.
As threat actor tactics evolve, our commitment to customer security remains unwavering. Together, we are building a more secure cloud environment, allowing you to innovate with confidence.
If you ever suspect unauthorized activity, please don’t hesitate to contact AWS Support immediately.
This year we will continue to pursue our commitment to improving the security of the Web PKI by introducing the option to get certificates with six-day lifetimes (“short-lived certificates”). We will also add support for IP addresses in addition to domain names. Our longer-lived certificates, which currently have a lifetime of 90 days, will continue to be available alongside our six-day offering. Subscribers will be able to opt in to short-lived certificates via a certificate profile mechanism being added to our ACME API.
Shorter Certificate Lifetimes Are Good for Security
When the private key associated with a certificate is compromised, the recommendation has always been to have the certificate revoked so that people will know not to use it. Unfortunately, certificate revocation doesn’t work very well. This means that certificates with compromised keys (or other issues) may continue to be used until they expire. The longer the lifetime of the certificate, the longer the potential for use of a problematic certificate.
The primary advantage of short-lived certificates is that they greatly reduce the potential compromise window because they expire relatively quickly. This reduces the need for certificate revocation, which has historically been unreliable. Our six-day certificates will not include OCSP or CRL URLs. Additionally, short-lived certificates practically require automation, and we believe that automating certificate issuance is important for security.
IP Address Support For Securing Additional Use Cases
We will support including IP addresses as Subject Alternative Names in our six-day certificates. This will enable secure TLS connections, with publicly trusted certificates, to services made available via IP address, without the need for a domain name.
Validation for IP addresses will work much the same as validation for domain names, though validation will be restricted to the http-01 and tls-alpn-01 challenge types. The dns-01 challenge type will not be available because the DNS is not involved in validating IP addresses. Additionally, there is no mechanism to check CAA records for IP addresses.
Timeline
We expect to issue the first valid short-lived certificates to ourselves in February of this year. Around April we will enable short-lived certificates for a small set of early adopting subscribers. We hope to make short-lived certificates generally available by the end of 2025.
The earliest short-lived certificates we issue may not support IP addresses, but we intend to enable IP address support by the time short-lived certificates reach general availability.
How To Get Six-Day and IP Address Certificates
Once short-lived certificates are an option for you, you’ll need to use an ACME client that supports ACME certificate profiles and select the short-lived certificate profile (the name of which will be published at a later date).
Once IP address support is an option for you, requesting an IP address in a certificate will automatically select a short-lived certificate profile.
Looking Ahead
The best way to prepare to take advantage of short-lived certificates is to make sure your ACME client is reliably renewing certificates in an automated fashion. If that’s working well then there should be no costs to switching to short-lived certificates.
If you have questions or comments about our plans, feel free to let us know on our community forums.
This year we will continue to pursue our commitment to improving the security of the Web PKI by introducing the option to get certificates with six-day lifetimes (“short-lived certificates”). We will also add support for IP addresses in addition to domain names. Our longer-lived certificates, which currently have a lifetime of 90 days, will continue to be available alongside our six-day offering. Subscribers will be able to opt in to short-lived certificates via a certificate profile mechanism being added to our ACME API.
Shorter Certificate Lifetimes Are Good for Security
When the private key associated with a certificate is compromised, the recommendation has always been to have the certificate revoked so that people will know not to use it. Unfortunately, certificate revocation doesn’t work very well. This means that certificates with compromised keys (or other issues) may continue to be used until they expire. The longer the lifetime of the certificate, the longer the potential for use of a problematic certificate.
The primary advantage of short-lived certificates is that they greatly reduce the potential compromise window because they expire relatively quickly. This reduces the need for certificate revocation, which has historically been unreliable. Our six-day certificates will not include OCSP or CRL URLs. Additionally, short-lived certificates practically require automation, and we believe that automating certificate issuance is important for security.
IP Address Support For Securing Additional Use Cases
We will support including IP addresses as Subject Alternative Names in our six-day certificates. This will enable secure TLS connections, with publicly trusted certificates, to services made available via IP address, without the need for a domain name.
Validation for IP addresses will work much the same as validation for domain names, though validation will be restricted to the http-01 and tls-alpn-01 challenge types. The dns-01 challenge type will not be available because the DNS is not involved in validating IP addresses. Additionally, there is no mechanism to check CAA records for IP addresses.
Timeline
We expect to issue the first valid short-lived certificates to ourselves in February of this year. Around April we will enable short-lived certificates for a small set of early adopting subscribers. We hope to make short-lived certificates generally available by the end of 2025.
The earliest short-lived certificates we issue may not support IP addresses, but we intend to enable IP address support by the time short-lived certificates reach general availability.
How To Get Six-Day and IP Address Certificates
Once short-lived certificates are an option for you, you’ll need to use an ACME client that supports ACME certificate profiles and select the short-lived certificate profile (the name of which will be published at a later date).
Once IP address support is an option for you, requesting an IP address in a certificate will automatically select a short-lived certificate profile.
Looking Ahead
The best way to prepare to take advantage of short-lived certificates is to make sure your ACME client is reliably renewing certificates in an automated fashion. If that’s working well then there should be no costs to switching to short-lived certificates.
If you have questions or comments about our plans, feel free to let us know on our community forums.
The Ghostty terminal emulator
project has generated a surprising amount of interest, even before
code was released to the public. This is in part due to the high
profile of its creator, HashiCorp founder
Mitchell Hashimoto. Its development was conducted behind closed doors
for beta testing, until version 1.0 was released
on December 26 under the MIT
license. While far from finished, Ghostty is ready for day-to-day
use and might be of interest to those who spend significant amounts of
time at the command line.
This post is co-written by Dr. Leonard Heilig and Meliena Zlotos from EUROGATE.
For container terminal operators, data-driven decision-making and efficient data sharing are vital to optimizing operations and boosting supply chain efficiency. Internally, making data accessible and fostering cross-departmental processing through advanced analytics and data science enhances information use and decision-making, leading to better resource allocation, reduced bottlenecks, and improved operational performance. Externally, sharing real-time data with partners such as shipping lines, trucking companies, and customs agencies fosters better coordination, visibility, and faster decision-making across the logistics chain. Together, these capabilities enable terminal operators to enhance efficiency and competitiveness in an industry that is increasingly data driven.
EUROGATE is a leading independent container terminal operator in Europe, known for its reliable and professional container handling services. Every day, EUROGATE handles thousands of freight containers moving in and out of ports as part of global supply chains. Their terminal operations rely heavily on seamless data flows and the management of vast volumes of data. Recently, EUROGATE has developed a digital twin for its container terminal Hamburg (CTH), generating millions of data points every second from Internet of Things (IoT)devices attached to its container handling equipment (CHE).
In this post, we show you how EUROGATE uses AWS services, including Amazon DataZone, to make data discoverable by data consumers across different business units so that they can innovate faster. Two use cases illustrate how this can be applied for business intelligence (BI) and data science applications, using AWS services such as Amazon Redshift and Amazon SageMaker. We encourage you to read Amazon DataZone concepts and terminology to become familiar with the terms used in this post.
Data landscape in EUROGATE and current challenges faced in data governance
The EUROGATE Group is a conglomerate of container terminals and service providers, providing container handling, intermodal transports, maintenance and repair, and seaworthy packaging services. In recent years, EUROGATE has made significant investments in modern cloud applications to enhance its operations and services along the logistics chains. With the addition of these technologies alongside existing systems like terminal operating systems (TOS) and SAP, the number of data producers has grown substantially. However, much of this data remains siloed and making it accessible for different purposes and other departments remains complex. Thus, managing data at scale and establishing data-driven decision support across different companies and departments within the EUROGATE Group remains a challenge.
Need for a data mesh architecture
Because entities in the EUROGATE group generate vast amounts of data from various sources—across departments, locations, and technologies—the traditional centralized data architecture struggles to keep up with the demands for real-time insights, agility, and scalability. The following requirements were essential to decide for adopting a modern data mesh architecture:
Domain-oriented ownership and data-as-a-product: EUROGATE aims to:
Enable scalable and straightforward data sharing across organizational boundaries.
Enhance agility by localizing changes within business domains and clear data contracts.
Improve accuracy and resiliency of analytics and machine learning by fostering data standards and high-quality data products.
Eliminate centralized bottlenecks and complex data pipelines.
Self-service and data governance: EUROGATE wants to ensure that the discovery, access, and use of data by consumers is as direct as possible through a data portal where information about shared data sets can be published, while data governance is streamlined through automated policy enforcement, ensuring compliance during key stages such as data discovery, access, and deployment.
Plug-and-play integration: A seamless, plug-and-play integration between data producers and consumers should facilitate rapid use of new data sets and enable quick proof of concepts, such as in the data science teams.
How Amazon DataZone helped EUROGATE address those challenges
In the first phase of establishing a data mesh, EUROGATE focused on standardized processes to allow data producers to share data in Amazon DataZone and to allow data consumers to discover and access data. The vision, as shown in the following figure, is that data from digital services, such as from the terminal operating system (TOS) and TwinSim (a project to create a digital twin of real-world operations), can be shared with Amazon DataZone and used by BI dashboards and data science teams, among others, while those digital services and other domain users can also consume subscribed data from Amazon DataZone.
In the following section, two use cases demonstrate how the data mesh is established with Amazon DataZone to better facilitate machine learning for an IoT-based digital twin and BI dashboards and reporting using Tableau.
Use case 1: Machine learning for IoT-based digital twin
Through the TwinSim project, EUROGATE has developed a digital twin using AWS services that gathers real-time data (for example, positions, machinery, and pick/deck events) from CHE (including straddle carriers and quay cranes), integrates it with planning data from the TOS, and enhances it with additional sources such as weather information. In addition to real-time analytics and visualization, the data needs to be shared for long-term data analytics and machine learning applications. EUROGATE’s data science team aims to create machine learning models that integrate key data sources from various AWS accounts, allowing for training and deployment across different container terminals. To achieve this, EUROGATE designed an architecture that uses Amazon DataZone to publish specific digital twin data sets, enabling access to them with SageMaker in a separate AWS account.
As part of the required data, CHE data is shared using Amazon DataZone. The data originates in Amazon Kinesis Data Streams, from which it is copied to a dedicated Amazon Simple Storage Service (Amazon S3) bucket by using Amazon Data Firehose in combination with an AWS Lambda function for data filtering. An extract, transform, and load (ETL) process using AWS Glue is triggered once a day to extract the required data and transform it into the required format and quality, following the data product principle of data mesh architectures. From here, the metadata is published to Amazon DataZone by using AWS Glue Data Catalog. This process is shown in the following figure.
To work with the shared data, the data science and AI teams subscribe to the data and query it using Amazon Athena by using Amazon SageMaker Data Wrangler. The following is an example query.
import awswrangler as wr
wr.athena.read_sql_query('SELECT * FROM "sagemakedatalakeenvironment_sub_db"."cycle_end"', "sagemakedatalakeenvironment_sub_db", ctas_approach=False)
A similar approach is used to connect to shared data from Amazon Redshift, which is also shared using Amazon DataZone.
import awswrangler as wr
con = wr.redshift.connect(secret_id="ai-dev-redshift-credentials",is_serverless=True,serverless_work_group="ai-dev-workgroup")
with con.cursor() as cursor:
cursor.execute('SELECT * FROM
"datazone_datashare_db_269e5790f589258657fcc48d8cfd65ea3f3cd7f7"."datazone_env_twinsimsilverdata"."cycle_end";')
con.close()
With this, as the data lands in the curated data lake (Amazon S3 in parquet format) in the producer account, the data science and AI teams gain instant access to the source data eliminating traditional delays in the data availability. The data science and AI teams are able to explore and use new data sources as they become available through Amazon DataZone. Because Amazon DataZone integrates the data quality results, by subscribing to the data from Amazon DataZone, the teams can make sure that the data product meets consistent quality standards.
After experimentation, the data science teams can share their assets and publish their models to an Amazon DataZone business catalog using the integration between Amazon SageMaker and Amazon DataZone. This will be the future use case of EUROGATE where the ability to publish trained machine learning (ML) models back to an Amazon DataZone catalog promotes reusability, allowing models to be discovered by other teams and projects. This approach fosters knowledge sharing across the ML lifecycle.
Use case 2: BI for cloud applications
In recent years, EUROGATE has developed several cloud applications for supporting key container logistics processes and services, such as special container terminal and container depot applications or digital platforms for organizing container transports using rail and truck. The applications are hosted in dedicated AWS accounts and require a BI dashboard and reporting services based on Tableau. In the past, one-to-one connections were established between Tableau and respective applications. This led to a complex and slow computations. In this use case, EUROGATE implemented a hybrid data mesh architecture using Amazon Redshift as a centralized data platform. This approach transformed their fragmented Tableau connections into a scalable, efficient analytics ecosystem.
By centralizing container and logistics application data through Amazon Redshift and establishing a governance framework with Amazon DataZone, EUROGATE achieved both performance optimization and cost efficiency. The hybrid data mesh enables batch processing at scale while maintaining the data access controls, security, and governance; effectively balancing the distributed ownership with centralized analytics capabilities.
The data is shared from on-premises to an Amazon Relational Database Service (Amazon RDS) database in the AWS Cloud. AWS Database Migration Service (AWS DMS) is used to securely transfer the relevant data to a central Amazon Redshift cluster. AWS DMS tasks are orchestrated using AWS Step Functions. A Step Functions state machine is run on a daily using Amazon EventBridge scheduler. The data in the central data warehouse in Amazon Redshift is then processed for analytical needs and the metadata is shared to the consumers through Amazon DataZone. The consumer subscribes to the data product from Amazon DataZone and consumes the data with their own Amazon Redshift instance. This is further integrated into Tableau dashboards. The architecture is depicted in the following figure.
Implementation benefits
As we continue to scale, efficient and seamless data sharing across services and applications becomes increasingly important. By using Amazon DataZone and other AWS services including Amazon Redshift and Amazon SageMaker, we can achieve a secure, streamlined, and scalable solution for data and ML model management, fostering effective collaboration and generating valuable insights. This approach supports both the immediate needs of visualization tools such as Tableau and the long-term demands of digital twin and IoT data analytics.
Centralized, scalable data sharing and native integration
Amazon DataZone facilitates integration with applications such as Tableau, enabling data to flow seamlessly within the AWS ecosystem. Those integrations reduce the need for complex, manual configurations, allowing EUROGATE to share data across the organization efficiently. The architecture centralizes key data, such as CHE data, for analytics and ML, ensuring that teams across the organization have access to consistent, up-to-date information, enhancing collaboration and decision-making at all levels. Insights from ML models can be channeled through Amazon DataZone to inform internal key decision makers internally and external partners.
Reduced complexity, greater scalability, and cost efficiency
The Amazon DataZone architecture reduces unnecessary complexity and scales with EUROGATE’s growing needs, whether through new data sources or increased user demand. In parallel, using Amazon Data Firehose to stream data into an S3 bucket and AWS Glue for daily ETL transformations provides an automated pipeline that prepares the data for long-term analytics. This batch-oriented approach reduces computational overhead and associated costs, allowing resources to be allocated efficiently. While real-time data is processed by other applications, this setup maintains high-performance analytics without the expense of continuous processing.
Faster and easier data integration for Tableau and enhanced data preparation for ML
Amazon DataZone streamlines data integration for tools such as Tableau, enabling BI teams to quickly add and visualize data without building complex pipelines. This agility accelerates EUROGATE’s insight generation, keeping decision-making aligned with current data. Additionally, daily ETL transformations through AWS Glue ensure high-quality, structured data for ML, enabling efficient model training and predictive analytics. This combination of ease and depth in data management equips EUROGATE to support both rapid BI needs and robust analytical processing for IoT and digital twin projects.
Faster onboarding and data sharing of data assets between organizational units
Amazon DataZone helps the teams to autonomously discover data assets that are created in the organization and to onboard data assets across AWS accounts within minutes with metadata synchronization. EUROGATE has already onboarded 500 data assets from different organizational units using Amazon DataZone. The new process of onboarding data assets is 15 times faster, leading to immediate visibility of data assets while simplifying data sharing and discovery through an intuitive point-and-click interface that removes traditional barriers to data access.
Conclusion
The implementation of Amazon DataZone marks a transformative step for EUROGATE’s data management by providing a scalable, and efficient solution for data sharing, machine learning and analytics. By integrating various data producers and connecting them to data consumers such as Amazon SageMaker and Tableau, Amazon DataZone functions as a digital library to streamline data sharing and integration across EUROGATE’s operations. In the first phase of production, Amazon DataZone has already demonstrated measurable benefits, including access to data and ML and the ability to incorporate a wider range of datasets to its unified catalog repository. By centralizing metadata with Amazon DataZone, EUROGATE is setting a solid foundation for efficient operations and improved data and ML governance, because teams can now discover, govern, and analyze data with greater confidence and speed. This capability supports rapid responses to business needs, helping EUROGATE to maintain agility and stay ahead of the curve. With this, EUROGATE is better positioned to onboard new data sources, integrate additional terminals, and expand machine learning applications across our container terminals.
Amazon DataZone empowers EUROGATE by setting the stage for long-term operational excellence and scalability. With a unified catalog, enhanced analytics capabilities, and efficient data transformation processes, we’re laying the groundwork for future growth. This infrastructure enables EUROGATE to extract predictive insights, drive smarter business decisions, and scale operations efficiently, ultimately supporting our goal of sustained innovation and competitive advantage.
Future vision and next steps
As EUROGATE continues to advance its digital transformation, the integration of Amazon DataZone and EUROGATE’s architecture lays the groundwork for a more data-driven and intelligent future. In the upcoming phases, the vision is to further expand the role of Amazon DataZone as the central platform for all data management, enabling seamless integration across an even broader set of data sources and consumers. This will include additional data from more container terminals and logistics service providers, enhanced operational metrics, IoT sensor data, and advanced third-party sources such as global supply chain data and maritime analytics.
The continued focus on secure data sharing and governance will also foster better collaboration with partners, suppliers, and customers, leading to improved service levels and a more resilient supply chain. This future vision will help EUROGATE maintain its position as a leader in container terminal operations while continuously adapting to technological advancements and market dynamics.
Ultimately, EUROGATE’s investment in this architecture ensures that the organization is well-positioned to scale and innovate in a dynamic industry through a future of smarter, more connected, and highly efficient container terminal operations.
To learn more about Amazon DataZone and how to get started, see the Getting started guide. See the YouTube playlist for some of the latest demos of Amazon DataZone and short descriptions of the capabilities available.
About the Authors
Dr. Leonard Heilig is CTO at driveMybox and drives digitalization and AI initiatives at EUROGATE, bringing over 10 years of research and industry experience in cloud-based platform development, data management, and AI. Combining a deep understanding of advanced technologies with a passion for innovation, Leonard is dedicated to transforming logistics processes through digitalization and AI-driven solutions.
Meliena Zlotos is a DevOps Engineer at EUROGATE with a background in Industrial Engineering. She has been heavily involved in the Data Sharing Project, focusing on the implementation of Amazon DataZone into EUROGATE’s IT environment. Through this project, Meliena has gained valuable experience and insights into DataZone and Data Engineering, contributing to the successful integration and optimization of data management solutions within the organization.
Lakshmi Nair is a Senior Specialist Solutions Architect for Data Analytics at AWS. She focuses on architecting solutions for organizations across their end-to-end data analytics estate, including batch and real-time streaming, data governance, big data, data warehousing, and data lake workloads. She can reached via LinkedIn.
Siamak Nariman is a Senior Product Manager at AWS. He is focused on AI/ML technology, ML model management, and ML governance to improve overall organizational efficiency and productivity. He has extensive experience automating processes and deploying various technologies.
Version
11.0.0 of the libvirt virtualization
API has been released. Notable changes in this release include
the ability to export virtiofs filesystems in
read-only mode, the addition of support for vlan tagging and trunking
of network interfaces with the network, qemu, and lxc drivers, as well
as a number of bug fixes.
We have just now received word of the passing
of Helen Borrie, a longtime contributor to the Firebird relational
database project.
Helen’s quiet leadership and dedication left a lasting impact on
Firebird and its users. Her efforts helped build not just a powerful
database but also a strong, collaborative community. She will be
deeply missed by all who knew her and benefited from her work.
She will be greatly missed. (Thanks to Steve Friedl.)
Linux Mint version
22.1, a long-term-support (LTS) release with support until 2029, is now
available. Notable changes in this release include a transition to Aptkit for background
package management tasks, Captain to install
Debian packages, and a new default theme with improved Wayland
compatibility. See the release notes for
known issues.
Nick Tait announced on the
oss-security mailing list that
rsync, the widely used file transfer program, had a number of serious vulnerabilities.
Users can mitigate all six vulnerabilities by upgrading to
version 3.4.0, which was
released on January 14. While all users should upgrade, servers that use rsyncd are
especially impacted:
In the most severe CVE, an attacker only requires
anonymous read access to a rsync server, such as a public mirror, to
execute arbitrary code on the machine the server is running on.
The collective thoughts of the interwebz
Manage Consent
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
