Де са ти, Гешев, осъдените олигарси?

Post Syndicated from Венелина Попова original https://toest.bg/de-sa-ti-geshev-osudenite-oligarsi/

На 24 март в онлайн пресконференция от Дубай обвиненият за 19 престъпления бизнесмен Васил Божков заяви, че е готов да се върне в България, при условие че прокуратурата свали неговото издирване от червената бюлетина на Интерпол. Ако се абстрахираме от множеството интерпретации по темата и без да навлизаме в правния ѝ аспект, съвсем логично е да се запитаме:

иска ли наистина държавното обвинение да разпита Васил Божков, или предпочита той да остане извън страната?

Името на знаковия хазартен бос нашумя отново след арестите на Бойко Борисов, Владислав Горанов и Севдeлина Арнаудова, извършени от полицията въз основа на данните, съдържащи се в сигнала на Божков до прокуратурата отпреди две години. Той бе изпратен още до президента, парламента, Висшия съдебен съвет, Върховния касационен съд, политическите партии в България и посолствата на Великобритания и САЩ у нас. В него бизнесменът твърди, че е дал около 60 млн. лв. на бившия премиер и на неговия финансов министър под формата на такса „спокойствие“.

Божков твърди, че не знае как прокуратурата е интерпретирала фактите в сигнала, прехвърлян през годините между Върховната касационна прокуратура, Специализираната прокуратура и данъчните служби, които е трябвало да проверят къде са отивали парите, теглени от него от банковите му сметки. „Каквито и данни да съм дал в тази прокуратура, те изчезват“, заяви на пресконференцията хазартният бос, който още с пускането на сигнала обяви, че разполага с достатъчно доказателствен материал, в т.ч. документи, банкови извлечения, записи от охранителни камери, снимки и свидетелски показания. Но на пресконференцията в четвъртък отказа да даде повече подробности за тях.

Арестите на Борисов, Горанов и Арнаудова, които Васил Божков нарече пред журналистите не просто организирана престъпна група, а директно „хунта и мафия“, нагнетиха напрежение в обществото и между институциите, а от ГЕРБ обявиха поход за сваляне на правителството и за нови избори. 24-часовият престой в ареста на лидера на най-голямата опозиционна партия в България имаше и международен отзвук, каквато беше и една от целите на задържането му под стража. Председателят на ЕНП в Европейския парламент Манфред Вебер заяви в сряда, че Борисов е бил „отвлечен“, и поиска проверка на законността на задържането му. Два дни по-рано групата на Прогресивния алианс на социалистите и демократите в ЕП пък излезе с позиция, в която призовава правосъдните власти в София и Брюксел да дадат отговор на въпросите, свързани с управлението на Борисов, и особено след като Европейската прокуратура потвърди, че е получила множество данни със съмнения за корупция и злоупотреби с европейски средства.

В хода на развитието на събитията след излизането на Борисов, Горанов и Арнаудова от ареста беше поставен и въпросът за това

може ли Васил Божков да бъде разпитан дистанционно от Дубай.

Според публично изразени мнения на юристи, сред тях и на министъра на вътрешните работи Бойко Рашков, процесуални пречки за подобен разпит няма. Очевидно не е имало пречка и пред Европейската прокуратура, която, по думите на Божков, го е открила и разпитала на 8 март т.г. от столицата на Обединените арабски емирства. Подобен разпит, само че в писмена форма, е извършен и от Дирекция „Национална полиция“, обяви той на пресконференцията. Пак от него разбрахме, че е осъществил контакти и с американските власти, на които е дал много повече сведения, отколкото на ЕП, за да бъде изваден от списъка „Магнитски“, попаднал в него „погрешно“. И какво се оказа накрая – че единствено за прокуратурата на Иван Гешев има процесуални пречки Васил Божков да бъде разпитан дистанционно. А хипотезата прокурори да разпитат бизнесмена в Дубай дори не е обсъждана публично.

Васил Божков, с прякор Черепа, е сочен за един от най-богатите олигарси в България, а във вътрешна кореспонденция на американския Държавен департамент е наричан и „най-печално известният гангстер на България“. Дали е бил съучастник на „шайката“, както той назовава хората около бившия премиер Борисов, които са го рекетирали и на които е плащал, за да развива успешно хазартния си бизнес, може да каже само съдът. Както и дали 60-те милиона, които той посочва, че е дал като процент от печалбата на „Национална лотария“ в периода от 2017 до 2019 г. на същите тези лица, отговарят на състава на престъплението подкуп. Но без съмнение сюжетът, който Божков представя две години от Дубай пред медиите, не изглежда съшит с бели конци. В него освен Борисов, Горанов и Арнаудова са още Менда Стоянова, Данаил Кирилов, Валери Симеонов, Кирил Домусчиев, пачки от банкноти по 500 евро, които се събират в джоба на едно мъжко сако, и една (не)запалена пура.

Този сюжет може да бъде разплетен единствено от разследващите органи в България.

И прокуратурата – господарят на разследването, може и трябва да разпита Васил Божков. Още повече че той обяви готовност да се завърне в България, и постави за това едно-единствено условие. Което означава, че ще може да бъде разследван и за всички повдигнати му обвинения, част от които коментира пред медиите като несъстоятелни и без доказателства. Ако главният прокурор не организира разпит на Васил Божков, независимо как ще бъде осъществен, ще потвърди пред обществото и пред партньорите ни в Брюксел, че е избран на този пост, за да прикрива корупцията по високите етажи на властта, а не да я разкрива.

С действията си досега Иван Гешев демонстрира и двойни стандарти към родните ни олигарси и бизнесмени, като арестува и обвинява избирателно само онези, които по една или друга причина са станали неудобни за властта. Но до момента специализираното правосъдие, което е пред закриване, не е приключило нито едно от тези дела с осъдителна присъда. А някои от задържаните предявиха иск срещу главния прокурор и неговите заместници за нанесените им морални щети. Тези действия на Гешев дават достатъчна легитимност на управляващите да искат неговата оставка. А зад тях е силата на омерзеното общество, очакващо справедливост.

Заглавна снимка: Стопкадър от онлайн пресконференцията на Васил Божков от 24 март т.г., излъчена на неговата Facebook страница

Източник

Insights for CTOs: Part 3 – Growing your business with modern data capabilities

Post Syndicated from Syed Jaffry original https://aws.amazon.com/blogs/architecture/insights-for-ctos-part-3-growing-your-business-with-modern-data-capabilities/

This post was co-wrtiten with Jonathan Hwang, head of Foundation Data Analytics at Zendesk.

In my role as a Senior Solutions Architect, I have spoken to chief technology officers (CTOs) and executive leadership of large enterprises like big banks, software as a service (SaaS) businesses, mid-sized enterprises, and startups.

In this 6-part series, I share insights gained from various CTOs and engineering leaders during their cloud adoption journeys at their respective organizations. I have taken these lessons and summarized architecture best practices to help you build and operate applications successfully in the cloud. This series also covers building and operating cloud applications, security, cloud financial management, modern data and artificial intelligence (AI), cloud operating models, and strategies for cloud migration.

In Part 3, I’ve collaborated with the head of Foundation Analytics at Zendesk, Jonathan Hwang, to show how Zendesk incrementally scaled their data and analytics capabilities to effectively use the insights they collect from customer interactions. Read how Zendesk built a modern data architecture using Amazon Simple Storage Service (Amazon S3) for storage, Apache Hudi for row-level data processing, and AWS Lake Formation for fine-grained access control.

Why Zendesk needed to build and scale their data platform

Zendesk is a customer service platform that connects over 100,000 brands with hundreds of millions of customers via telephone, chat, email, messaging, social channels, communities, review sites, and help centers. They use data from these channels to make informed business decisions and create new and updated products.

In 2014, Zendesk’s data team built the first version of their big data platform in their own data center using Apache Hadoop for incubating their machine learning (ML) initiative. With that, they launched Answer Bot and Zendesk Benchmark report. These products were so successful they soon overwhelmed the limited compute resources available in the data center. By the end of 2017, it was clear Zendesk needed to move to the cloud to modernize and scale their data capabilities.

Incrementally modernizing data capabilities

Zendesk built and scaled their workload to use data lakes on AWS, but soon encountered new architecture challenges:

  • The General Data Protection Regulation (GDPR) “right to be forgotten” rule made it difficult and costly to maintain data lakes, because deleting a small piece of data required reprocessing large datasets.
  • Security and governance was harder to manage when data lake scaled to a larger number of users.

The following sections show you how Zendesk is addressing GDPR rules by evolving from plain Apache Parquet files on Amazon S3 to Hudi datasets on Amazon S3 to enable row level inserts/updates/deletes. To address security and governance, Zendesk is migrating to AWS Lake Formation centralized security for fine-grained access control at scale.

Zendesk’s data platform

Figure 1 shows Zendesk’s current data platform. It consists of three data pipelines: “Data Hub,” “Data Lake,” and “Self Service.”

Zendesk data pipelines

Figure 1. Zendesk data pipelines

Data Lake pipelines

The Data Lake and Data Hub pipelines cover the entire lifecycle of the data from ingestion to consumption.

The Data Lake pipelines consolidate the data from Zendesk’s highly distributed databases into a data lake for analysis.

Zendesk uses Amazon Database Migration Service (AWS DMS) for change data capture (CDC) from over 1,800 Amazon Aurora MySQL databases in eight AWS Regions. It detects transaction changes and applies them to the data lake using Amazon EMR and Hudi.

Zendesk ticket data consists of over 10 billion events and petabytes of data. The data lake files in Amazon S3 are transformed and stored in Apache Hudi format and registered on the AWS Glue catalog to be available as data lake tables for analytics querying and consumption via Amazon Athena.

Data Hub pipelines

The Data Hub pipelines focus on real-time events and streaming analytics use cases with Apache Kafka. Any application at Zendesk can publish events to a global Kafka message bus. Apache Flink ingests these events into Amazon S3.

The Data Hub provides high-quality business data that is highly available and scalable.

Self-managed pipeline

The self-managed pipelines empower product engineering teams to use the data lake for those use cases that don’t fit into our standard integration patterns. All internal Zendesk product engineering teams can use standard tools such as Amazon EMR, Amazon S3, Athena, and AWS Glue to publish their own analytics dataset and share them with other teams.

A notable example of this is Zendesk’s fraud detection engineering team. They publish their fraud detection data and findings through our self-manage data lake platform and use Amazon QuickSight for visualization.

You need fine-grained security and compliance

Data lakes can accelerate growth through faster decision making and product innovation. However, they can also bring new security and compliance challenges:

  • Visibility and auditability. Who has access to what data? What level of access do people have and how/when and who is accessing it?
  • Fine-grained access control. How do you define and enforce least privilege access to subsets of data at scale without creating bottlenecks or key person/team dependencies?

Lake Formation helps address these concerns by auditing data access and offering row- and column-level security and a delegated access control model to create data stewards for self-managed security and governance.

Zendesk used Lake Formation to build a fine-grained access control model that uses row-level security. It detects personally identifiable information (PII) while scaling the data lake for self-managed consumption.

Some Zendesk customers opt out of having their data included in ML or market research. Zendesk uses Lake Formation to apply row-level security to filter out records associated with a list of customer accounts who have opted out of queries. They also help data lake users understand which data lake tables contain PII by automatically detecting and tagging columns in the data catalog using AWS Glue’s PII detection algorithm.

The value of real-time data processing

When you process and consume data closer to the time of its creation, you can make faster decisions. Streaming analytics design patterns, implemented using services like Amazon Managed Streaming for Apache Kafka (Amazon MSK) or Amazon Kinesis, create an enterprise event bus to exchange data between heterogeneous applications in near real time.

For example, it is common to use streaming to augment the traditional database CDC ingestion into the data lake with additional streaming ingestion of application events. CDC is a common data ingestion pattern, but the information can be too low level. This requires application context to be reconstructed in the data lake and business logic to be duplicated in two places, inside the application and in the data lake processing layer. This creates a risk of semantic misrepresentation of the application context.

Zendesk faced this challenge with their CDC data lake ingestion from their Aurora clusters. They created an enterprise event bus built with Apache Kafka to augment their CDC with higher-level application domain events to be exchanged directly between heterogeneous applications.

Zendesk’s streaming architecture

A CDC database ticket table schema can sometimes contain unnecessary and complex attributes that are application specific and do not capture the domain model of the ticket. This makes it hard for downstream consumers to understand and use the data. A ticket domain object may span several database tables when modeled in third normal form, which makes querying for analysts difficult downstream. This is also a brittle integration method because downstream data consumers can easily be impacted when the application logic changes, which makes it hard to derive a common data view.

To move towards event-based communication between microservices, Zendesk created the Platform Data Architecture (PDA) project, which uses a standard object model to represent a higher level, semantic view of their application data. Standard objects are domain objects designed for cross-domain communication and do not suffer from the lower level fragmented scope of database CDC. Ultimately, Zendesk aims to transition their data architecture from a collection of isolated products and data silos into a cohesive unified data platform.

An application view of Zendesk’s streaming architecture

Figure 2. An application view of Zendesk’s streaming architecture

Figure 3 shows how all Zendesk products and users integrate through common standard objects and standard events within the Data Hub. Applications publish and consume standard objects and events to/from the event bus.

For example, a complete ticket standard object will be published to the message bus whenever it is created, updated, or changed. On the consumption side, these events get used by product teams to enable platform capabilities such as search, data export, analytics, and reporting dashboards.

Summary

As Zendesk’s business grew, their data lake evolved from simple Parquet files on Amazon S3 to a modern Hudi-based incrementally updateable data lake. Now, their original coarse-grained IAM security policies use fine-grained access control with Lake Formation.

We have repeatedly seen this incremental architecture evolution achieve success because it reduces the business risk associated with the change and provides sufficient time for your team to learn and evaluate cloud operations and managed services.

Looking for more architecture content? AWS Architecture Center provides reference architecture diagrams, vetted architecture solutions, Well-Architected best practices, patterns, icons, and more!

Other posts in this series

Horn: Racing against the clock

Post Syndicated from original https://lwn.net/Articles/889183/

Jann Horn describes
in great detail the process he went through to exploit a tiny race
window in the kernel.

Luckily for us, the race window contains the first few memory
accesses to the struct file; therefore, by making sure that the
struct file is not present in the fastest CPU caches, we can widen
the race window by as much time as the memory accesses take. The
standard way to do this is to use an eviction pattern / eviction
set; but instead we can also make the cache line dirty on another
core.

Ekstrand: How to write a Vulkan driver in 2022

Post Syndicated from original https://lwn.net/Articles/889176/

Over on the Collabora blog, Jason Ekstrand has a detailed look at writing a Vulkan graphics driver in today’s world. “Not only has Vulkan grown, but Mesa has as well, and we’ve built up quite a suite of utilities and helpers for making writing Vulkan drivers easier.” The blog post takes the form of a tutorial of sorts, though the end result is not a functioning Vulkan driver the framework of one is shown.

At the time we were developing ANV (the Intel Vulkan driver), the Vulkan spec itself was still under development and everything was constantly in flux. There were no best practices; there were barely even tools. Everyone working on Vulkan was making it up as they went because it was a totally new API. Most of the code we wrote was purpose-built for the Intel driver because there were no other Mesa drivers to share code. (Except for the short-lived LunarG Intel driver based in ilo, which we were replacing.) If we had tried to build abstractions, they could have gotten shot to pieces at any moment by a spec change. (We rewrote the descriptor set layout code from scratch at least five or six times before the driver ever shipped.) It was frustrating, exhausting, and a whole lot of fun.

These days, however, the Vulkan spec has been stable and shipping for six years, the tooling and testing situation is pretty solid, and there are six Vulkan drivers in the Mesa tree with more on the way. We’ve also built up a lot of common infrastructure. This is important both because it makes writing a Vulkan driver easier and because it lets us fix certain classes of annoying bugs in a common place instead of everyone copying and pasting those bugs.

AWS Lambda Now Supports Up to 10 GB Ephemeral Storage

Post Syndicated from Channy Yun original https://aws.amazon.com/blogs/aws/aws-lambda-now-supports-up-to-10-gb-ephemeral-storage/

Serverless applications are event-driven, using ephemeral compute functions ranging from web APIs, mobile backends, and streaming analytics to data processing stages in machine learning (ML) and high-performance applications. While AWS Lambda includes a 512 MB temporary file system (/tmp) for your code, this is an ephemeral scratch resource not intended for durable storage such as Amazon Elastic File System (Amazon EFS).

However, extract, transform, and load (ETL) jobs and content generation workflows such as creating PDF files or media transcoding require fast, scalable local storage to process large amounts of data quickly. Data-intensive applications require large amounts of temporary data specific to the invocation or cached data that can be reused for all invocation in the same execution environment in a highly performant manner. With the previous limit of 512 MB, customers had to selectively load data from Amazon Simple Storage Service (Amazon S3) and Amazon EFS, or increase the allocated function memory and thus increase their cost, just to handle large objects downloaded from Amazon S3. Since customers could not cache larger data locally in the Lambda execution environment, every function invoke had to read data in parallel, which made scaling out harder for customers.

Today, we are announcing that AWS Lambda now allows you to configure ephemeral storage (/tmp) between 512 MB and 10,240 MB. You can now control the amount of ephemeral storage a function gets for reading or writing data, allowing you to use AWS Lambda for ETL jobs, ML inference, or other data-intensive workloads.

With increased AWS Lambda ephemeral storage, you get access to a secure, low-latency ephemeral file system up to 10 GB. You can continue to use up to 512 MB for free and are charged for the amount of storage you configure over the free limit for the duration of invokes.

Setting Larger Ephemeral Storage for Your Lambda Function
To configure your Lambda function with larger ephemeral storage, choose the Configuration tab under the General Configuration section in the AWS Lambda Console. You will see a new configuration for Ephemeral storage setting at 512MB by default.

When you click the Edit button, you can configure the ephemeral storage from 512 MB to 10,240 MB in 1 MB increments for your Lambda functions.

With AWS Command Line Interface (AWS CLI), you can update your desired size of ephemeral storage using theupdate-function-configuration command.

$ aws lambda update-function-configuration --function-name PDFGenerator \
              --ephemeral-storage '{"Size": 10240}'

You can configure ephemeral storage using Lambda API via AWS SDK and AWS CloudFormation. To learn more, see Configuring function options in the AWS Documentation.

As a review, AWS Lambda provides a comprehensive range of storage options. To learn more, see a great blog post, Choosing between AWS Lambda data storage options in web apps, written by my colleague James Beswick. I want to quote the table to show the differences between these options and common use-cases to help you choose the right one for your own applications.

Features Ephemeral Storage (/tmp) Lambda Layers Amazon EFS Amazon S3
Maximum size 10,240 MB 50 MB (direct upload) Elastic Elastic
Persistence Ephemeral Durable Durable Durable
Content Dynamic Static Dynamic Dynamic
Storage type File system Archive File system Object
Lambda event source integration N/A N/A N/A Native
Operations supported Any file system operation Immutable Any file system operation Atomic with versioning
Object tagging and metadata
 N N N Y
Pricing model Included in Lambda
(Charged over 512MB)		 Included in Lambda Storage + data transfer + throughput Storage + requests + data transfer
Shared across all invocations N Y Y Y
Sharing/permissions model Function-only IAM IAM + NFS IAM
Source for AWS Glue and Amazon Quicksight
 N N N Y
Relative data access speed from Lambda Fastest Fastest Very fast Fast

Available Now
You can now configure up to 10 GB of ephemeral storage per Lambda function instance in all Regions where AWS Lambda is available. With 10 GB container image support, 10 GB function memory, and now 10 GB of ephemeral function storage, you can support workloads such as using large temporal files, data and media processing, machine learning inference, and financial analysis.

Support is also available through many AWS Lambda Partners such as HashiCorp (Terraform), Pulumi, Datadog, Splunk (SignalFx), Lumigo, Thundra, Dynatrace, Slalom, Cloudwiry, and Contino.

For this feature, you are charged for the storage you configure over the 512 MB free limit for the duration of your function invokes. To learn more, visit AWS Lambda product and pricing page and send feedback through the AWS re:Post for AWS Lambda or your usual AWS Support contacts.

– Channy

What to consider when migrating data warehouse to Amazon Redshift

Post Syndicated from Lewis Tang original https://aws.amazon.com/blogs/big-data/what-to-consider-when-migrating-data-warehouse-to-amazon-redshift/

Customers are migrating data warehouses to Amazon Redshift because it’s fast, scalable, and cost-effective. However, data warehouse migration projects can be complex and challenging. In this post, I help you understand the common drivers of data warehouse migration, migration strategies, and what tools and services are available to assist with your migration project.

Let’s first discuss the big data landscape, the meaning of a modern data architecture, and what you need to consider for your data warehouse migration project when building a modern data architecture.

Business opportunities

Data is changing the way we work, live, and play. All of this behavior change and the movement to the cloud has resulted in a data explosion over the past 20 years. The proliferation of Internet of Things and smart phones have accelerated the amount of the data that is generated every day. Business models have shifted, and so have the needs of the people running these businesses. We have moved from talking about terabytes of data just a few years ago to now petabytes and exabytes of data. By putting data to work efficiently and building deep business insights from the data collected, businesses in different industries and of various sizes can achieve a wide range of business outcomes. These can be broadly categorized into the following core business outcomes:

  • Improving operational efficiency – By making sense of the data collected from various operational processes, businesses can improve customer experience, increase production efficiency, and increase sales and marketing agility
  • Making more informed decisions – Through developing more meaningful insights by bringing together full picture of data across an organization, businesses can make more informed decisions
  • Accelerating innovation – Combining internal and external data sources enable a variety of AI and machine learning (ML) use cases that help businesses automate processes and unlock business opportunities that were either impossible to do or too difficult to do before

Business challenges

Exponential data growth has also presented business challenges.

First of all, businesses need to access all data across the organization, and data may be distributed in silos. It comes from a variety of sources, in a wide range of data types and in large volume and velocity. Some data may be stored as structured data in relational databases. Other data may be stored as semi-structured data in object stores, such as media files and the clickstream data that is constantly streaming from mobile devices.

Secondly, to build insights from data, businesses need to dive deep into the data by conducting analytics. These analytics activities generally involve dozens and hundreds of data analysts who need to access the system simultaneously. Having a performant system that is scalable to meet the query demand is often a challenge. It gets more complex when businesses need to share the analyzed data with their customers.

Last but not least, businesses need a cost-effective solution to address data silos, performance, scalability, security, and compliance challenges. Being able to visualize and predict cost is necessary for a business to measure the cost-effectiveness of its solution.

To solve these challenges, businesses need a future proof modern data architecture and a robust, efficient analytics system.

Modern data architecture

A modern data architecture enables organizations to store any amount of data in open formats, break down disconnected data silos, empower users to run analytics or ML using their preferred tool or technique, and manage who has access to specific pieces of data with the proper security and data governance controls.

The AWS data lake architecture is a modern data architecture that enables you to store data in a data lake and use a ring of purpose-built data services around the lake, as shown in the following figure. This allows you to make decisions with speed and agility, at scale, and cost-effectively. For more details, refer to Modern Data Architecture on AWS.

Modern data warehouse

Amazon Redshift is a fully managed, scalable, modern data warehouse that accelerates time to insights with fast, easy, and secure analytics at scale. With Amazon Redshift, you can analyze all your data and get performance at any scale with low and predictable costs.

Amazon Redshift offers the following benefits:

  • Analyze all your data – With Amazon Redshift, you can easily analyze all your data across your data warehouse and data lake with consistent security and governance policies. We call this the modern data architecture. With Amazon Redshift Spectrum, you can query data in your data lake with no need for loading or other data preparation. And with data lake export, you can save the results of an Amazon Redshift query back into the lake. This means you can take advantage of real-time analytics and ML/AI use cases without re-architecture, because Amazon Redshift is fully integrated with your data lake. With new capabilities like data sharing, you can easily share data across Amazon Redshift clusters both internally and externally, so everyone has a live and consistent view of the data. Amazon Redshift ML makes it easy to do more with your data—you can create, train, and deploy ML models using familiar SQL commands directly in Amazon Redshift data warehouses.
  • Fast performance at any scale – Amazon Redshift is a self-tuning and self-learning system that allows you to get the best performance for your workloads without the undifferentiated heavy lifting of tuning your data warehouse with tasks such as defining sort keys and distribution keys, and new capabilities like materialized views, auto-refresh, and auto-query rewrite. Amazon Redshift scales to deliver consistently fast results from gigabytes to petabytes of data, and from a few users to thousands. As your user base scales to thousands of concurrent users, the concurrency scaling capability automatically deploys the necessary compute resources to manage the additional load. Amazon Redshift RA3 instances with managed storage separate compute and storage, so you can scale each independently and only pay for the storage you need. AQUA (Advanced Query Accelerator) for Amazon Redshift is a new distributed and hardware-accelerated cache that automatically boosts certain types of queries.
  • Easy analytics for everyone – Amazon Redshift is a fully managed data warehouse that abstracts away the burden of detailed infrastructure management or performance optimization. You can focus on getting to insights, rather than performing maintenance tasks like provisioning infrastructure, creating backups, setting up the layout of data, and other tasks. You can operate data in open formats, use familiar SQL commands, and take advantage of query visualizations available through the new Query Editor v2. You can also access data from any application through a secure data API without configuring software drivers, managing database connections. Amazon Redshift is compatible with business intelligence (BI) tools, opening up the power and integration of Amazon Redshift to business users who operate from within the BI tool.

A modern data architecture with a data lake architecture and modern data warehouse with Amazon Redshift helps businesses in all different sizes address big data challenges, make sense of a large amount of data, and drive business outcomes. You can start the journey of building a modern data architecture by migrating your data warehouse to Amazon Redshift.

Migration considerations

Data warehouse migration presents a challenge in terms of project complexity and poses a risk in terms of resources, time, and cost. To reduce the complexity of data warehouse migration, it’s essential to choose a right migration strategy based on your existing data warehouse landscape and the amount of transformation required to migrate to Amazon Redshift. The following are the key factors that can influence your migration strategy decision:

  • Size – The total size of the source data warehouse to be migrated is determined by the objects, tables, and databases that are included in the migration. A good understanding of the data sources and data domains required for moving to Amazon Redshift leads to an optimal sizing of the migration project.
  • Data transfer – Data warehouse migration involves data transfer between the source data warehouse servers and AWS. You can either transfer data over a network interconnection between the source location and AWS such as AWS Direct Connect or transfer data offline via the tools or services such as the AWS Snow Family.
  • Data change rate – How often do data updates or changes occur in your data warehouse? Your existing data warehouse data change rate determines the update intervals required to keep the source data warehouse and the target Amazon Redshift in sync. A source data warehouse with a high data change rate requires the service switching from the source to Amazon Redshift to complete within an update interval, which leads to a shorter migration cutover window.
  • Data transformation – Moving your existing data warehouse to Amazon Redshift is a heterogenous migration involving data transformation such as data mapping and schema change. The complexity of data transformation determines the processing time required for an iteration of migration.
  • Migration and ETL tools – The selection of migration and extract, transform, and load (ETL) tools can impact the migration project. For example, the efforts required for deployment and setup of these tools can vary. We look closer at AWS tools and services shortly.

After you have factored in all these considerations, you can pick a migration strategy option for your Amazon Redshift migration project.

Migration strategies

You can choose from three migration strategies: one-step migration, two-step migration, or wave-based migration.

One-step migration is a good option for databases that don’t require continuous operation such as continuous replication to keep ongoing data changes in sync between the source and destination. You can extract existing databases as comma separated value (CSV) files, or columnar format like Parquet, then use AWS Snow Family services such as AWS Snowball to deliver datasets to Amazon Simple Storage Service (Amazon S3) for loading into Amazon Redshift. You then test the destination Amazon Redshift database for data consistency with the source. After all validations have passed, the database is switched over to AWS.

Two-step migration is commonly used for databases of any size that require continuous operation, such as the continuous replication. During the migration, the source databases have ongoing data changes, and continuous replication keeps data changes in sync between the source and Amazon Redshift. The breakdown of the two-step migration strategy is as follows:

  • Initial data migration – The data is extracted from the source database, preferably during non-peak usage to minimize the impact. The data is then migrated to Amazon Redshift by following the one-step migration approach described previously.
  • Changed data migration – Data that changed in the source database after the initial data migration is propagated to the destination before switchover. This step synchronizes the source and destination databases. After all the changed data is migrated, you can validate the data in the destination database and perform necessary tests. If all tests are passed, you then switch over to the Amazon Redshift data warehouse.

Wave-based migration is suitable for large-scale data warehouse migration projects. The principle of wave-based migration is taking precautions to divide a complex migration project into multiple logical and systematic waves. This strategy can significantly reduce the complexity and risk. You start from a workload that covers a good number of data sources and subject areas with medium complexity, then add more data sources and subject areas in each subsequent wave. With this strategy, you run both the source data warehouse and Amazon Redshift production environments in parallel for a certain amount of time before you can fully retire the source data warehouse. See Develop an application migration methodology to modernize your data warehouse with Amazon Redshift for details on how to identify and group data sources and analytics applications to migrate from the source data warehouse to Amazon Redshift using the wave-based migration approach.

To guide your migration strategy decision, refer to the following table to map the consideration factors with a preferred migration strategy.

. One-Step Migration Two-Step Migration Wave-Based Migration
The number of subject areas in migration scope Small Medium to Large Medium to Large
Data transfer volume Small to Large Small to Large Small to Large
Data change rate during migration None Minimal to Frequent Minimal to Frequent
Data transformation complexity Any Any Any
Migration change window for switching from source to target Hours Seconds Seconds
Migration project duration Weeks Weeks to Months Months

Migration process

In this section, we review the three high-level steps of the migration process. The two-step migration strategy and wave-based migration strategy involve all three migration steps. However, the wave-based migration strategy includes a number of iterations. Because only databases that don’t require continuous operations are good fits for one-step migration, only Steps 1 and 2 in the migration process are required.

Step 1: Convert schema and subject area

In this step, you make the source data warehouse schema compatible with the Amazon Redshift schema by converting the source data warehouse schema using schema conversion tools such as AWS Schema Conversion Tool (AWS SCT) and the other tools from AWS partners. In some situations, you may also be required to use custom code to conduct complex schema conversions. We dive deeper into AWS SCT and migration best practices in a later section.

Step 2: Initial data extraction and load

In this step, you complete the initial data extraction and load the source data into Amazon Redshift for the first time. You can use AWS SCT data extractors to extract data from the source data warehouse and load data to Amazon S3 if your data size and data transfer requirements allow you to transfer data over the interconnected network. Alternatively, if there are limitations such as network capacity limit, you can load data to Snowball and from there data gets loaded to Amazon S3. When the data in the source data warehouse is available on Amazon S3, it’s loaded to Amazon Redshift. In situations when the source data warehouse native tools do a better data unload and load job than AWS SCT data extractors, you may choose to use the native tools to complete this step.

Step 3: Delta and incremental load

In this step, you use AWS SCT and sometimes source data warehouse native tools to capture and load delta or incremental changes from sources to Amazon Redshift. This is often referred to change data capture (CDC). CDC is a process that captures changes made in a database, and ensures that those changes are replicated to a destination such as a data warehouse.

You should now have enough information to start developing a migration plan for your data warehouse. In the following section, I dive deeper into the AWS services that can help you migrate your data warehouse to Amazon Redshift, and the best practices of using these services to accelerate a successful delivery of your data warehouse migration project.

Data warehouse migration services

Data warehouse migration involves a set of services and tools to support the migration process. You begin with creating a database migration assessment report and then converting the source data schema to be compatible with Amazon Redshift by using AWS SCT. To move data, you can use the AWS SCT data extraction tool, which has integration with AWS Data Migration Service (AWS DMS) to create and manage AWS DMS tasks and orchestrate data migration.

To transfer source data over the interconnected network between the source and AWS, you can use AWS Storage Gateway, Amazon Kinesis Data Firehose, Direct Connect, AWS Transfer Family services, Amazon S3 Transfer Acceleration, and AWS DataSync. For data warehouse migration involving a large volume of data, or if there are constraints with the interconnected network capacity, you can transfer data using the AWS Snow Family of services. With this approach, you can copy the data to the device, send it back to AWS, and have the data copied to Amazon Redshift via Amazon S3.

AWS SCT is an essential service to accelerate your data warehouse migration to Amazon Redshift. Let’s dive deeper into it.

Migrating using AWS SCT

AWS SCT automates much of the process of converting your data warehouse schema to an Amazon Redshift database schema. Because the source and target database engines can have many different features and capabilities, AWS SCT attempts to create an equivalent schema in your target database wherever possible. If no direct conversion is possible, AWS SCT creates a database migration assessment report to help you convert your schema. The database migration assessment report provides important information about the conversion of the schema from your source database to your target database. The report summarizes all the schema conversion tasks and details the action items for schema objects that can’t be converted to the DB engine of your target database. The report also includes estimates of the amount of effort that it will take to write the equivalent code in your target database that can’t be converted automatically.

Storage optimization is the heart of a data warehouse conversion. When using your Amazon Redshift database as a source and a test Amazon Redshift database as the target, AWS SCT recommends sort keys and distribution keys to optimize your database.

With AWS SCT, you can convert the following data warehouse schemas to Amazon Redshift:

  • Amazon Redshift
  • Azure Synapse Analytics (version 10)
  • Greenplum Database (version 4.3 and later)
  • Microsoft SQL Server (version 2008 and later)
  • Netezza (version 7.0.3 and later)
  • Oracle (version 10.2 and later)
  • Snowflake (version 3)
  • Teradata (version 13 and later)
  • Vertica (version 7.2 and later)

At AWS, we continue to release new features and enhancements to improve our product. For the latest supported conversions, visit the AWS SCT User Guide.

Migrating data using AWS SCT data extraction tool

You can use an AWS SCT data extraction tool to extract data from your on-premises data warehouse and migrate it to Amazon Redshift. The agent extracts your data and uploads the data to either Amazon S3 or, for large-scale migrations, an AWS Snowball Family service. You can then use AWS SCT to copy the data to Amazon Redshift. Amazon S3 is a storage and retrieval service. To store an object in Amazon S3, you upload the file you want to store to an S3 bucket. When you upload a file, you can set permissions on the object and also on any metadata.

In large-scale migrations involving data upload to a AWS Snowball Family service, you can use wizard-based workflows in AWS SCT to automate the process in which the data extraction tool orchestrates AWS DMS to perform the actual migration.

Considerations for Amazon Redshift migration tools

To improve and accelerate data warehouse migration to Amazon Redshift, consider the following tips and best practices. Tthis list is not exhaustive. Make sure you have a good understanding of your data warehouse profile and determine which best practices you can use for your migration project.

  • Use AWS SCT to create a migration assessment report and scope migration effort.
  • Automate migration with AWS SCT where possible. The experience from our customers shows that AWS SCT can automatically create the majority of DDL and SQL scripts.
  • When automated schema conversion is not possible, use custom scripting for the code conversion.
  • Install AWS SCT data extractor agents as close as possible to the data source to improve data migration performance and reliability.
  • To improve data migration performance, properly size your Amazon Elastic Compute Cloud (Amazon EC2) instance and its equivalent virtual machines that the data extractor agents are installed on.
  • Configure multiple data extractor agents to run multiple tasks in parallel to improve data migration performance by maximizing the usage of the allocated network bandwidth.
  • Adjust AWS SCT memory configuration to improve schema conversion performance.
  • Use Amazon S3 to store the large objects such as images, PDFs, and other binary data from your existing data warehouse.
  • To migrate large tables, use virtual partitioning and create sub-tasks to improve data migration performance.
  • Understand the use cases of AWS services such as Direct Connect, the AWS Transfer Family, and the AWS Snow Family. Select the right service or tool to meet your data migration requirements.
  • Understand AWS service quotas and make informed migration design decisions.

Summary

Data is growing in volume and complexity faster than ever. However, only a fraction of this invaluable asset is available for analysis. Traditional on-premises data warehouses have rigid architectures that don’t scale for modern big data analytics use cases. These traditional data warehouses are expensive to set up and operate, and require large upfront investments in both software and hardware.

In this post, we discussed Amazon Redshift as a fully managed, scalable, modern data warehouse that can help you analyze all your data, and achieve performance at any scale with low and predictable cost. To migrate your data warehouse to Amazon Redshift, you need to consider a range of factors, such as the total size of the data warehouse, data change rate, and data transformation complexity, before picking a suitable migration strategy and process to reduce the complexity and cost of your data warehouse migration project. With AWS services such AWS SCT and AWS DMS, and by adopting the tips and the best practices of these services, you can automate migration tasks, scale migration, accelerate the delivery of your data warehouse migration project, and delight your customers.

About the Author

Lewis Tang is a Senior Solutions Architect at Amazon Web Services based in Sydney, Australia. Lewis provides partners guidance to a broad range of AWS services and help partners to accelerate AWS practice growth.

4 Fallacies That Keep SMBs Vulnerable to Ransomware, Pt. 1

Post Syndicated from Ryan Weeks original https://blog.rapid7.com/2022/03/24/four-fallacies-that-keep-smbs-vulnerable-to-ransomware-pt-1/

4 Fallacies That Keep SMBs Vulnerable to Ransomware, Pt. 1

Ransomware has focused on big-game hunting of large enterprises in the past years, and those events often make the headlines. The risk can be even more serious for small and medium-sized businesses (SMBs), who struggle to both understand the changing nature of the threats and lack the resources to become cyber resilient. Ransomware poses a greater threat to SMBs’ core ability to continue to operate, as recovery can be impossible or expensive beyond their means.

SMBs commonly seek assistance from managed services providers (MSPs) for their foundational IT needs to run their business — MSPs have been the virtual CIOs for SMBs for years. Increasingly, SMBs are also turning to their MSP partners to help them fight the threat of ransomware, implicitly asking them to also take on the role of a virtual CISO, too. These MSPs have working knowledge of ransomware and are uniquely situated to assist SMBs that are ready to go on a cyber resilience journey.

With this expert assistance available, one would think that we would be making more progress on ransomware. However, MSPs are still meeting resistance when working to implement a cyber resilience plan for many SMBs.

In our experience working with MSPs and hearing the challenges they face with SMBs, we have come to the conclusion that much of this resistance they meet is based on under-awareness, biases, or fallacies.

In this two-part blog series, we will present four common mistakes SMBs make when thinking about ransomware risk, allowing you to examine your own beliefs and draw new conclusions. We contend that until SMBs resistance to resilience improvement do the work to unwind critical flaws in thinking, ransomware will continue to be a growing and existential problem they face.

1. Relying on flawed thinking

“I’m concerned about the potential impacts of ransomware, but I do not have anything valuable that an attacker would want, so ransomware is not likely to happen to me.”

Formal fallacies

These arguments are the most common form of resistance toward implementing adequate cyber resilience for SMBs, and they create a rationalization for inaction as well as a false sense of safety. However, they are formal fallacies, relying on common beliefs that are partially informed by cognitive biases.

Formal fallacies can best be classified simply as deductively invalid arguments that typically commit an easily recognizable logical error when properly examined. Either the premises are untrue, or the argument is invalid due to a logical flaw.

Looking at this argument, the conclusion “ransomware will not happen to me” is the logical conclusion of the prior statement, “I have nothing of value to an attacker.” The flaw in this argument is that the attacker does not need the data they steal or hold ransom to be intrinsically valuable to them — they only need it to be valuable to the attack target.

Data that is intrinsically valuable is nice to have for an attacker, as they can monetize it outside of the attack by exfiltrating it and selling it (potentially multiple times), but the primary objective is to hold it ransom, because you need it to run your business. Facing this fact, we can see that the conclusion “ransomware will not happen to me” is logically invalid based on the premise “I have nothing of value to an attacker.”

Confirmation bias

The belief “ransomware will not happen to me” can also be a standalone argument. The challenge here is that the premise of the argument is unknown. This means we need data to support probability. With insufficient reporting data to capture accurate rates of ransomware on SMBs, this is problematic and can lead to confirmation bias. If I can’t find data on others like me as an SMB, then I may conclude that this confirms I’m not at risk.

Anchoring bias

I may be able to find data in aggregate that states that my SMB’s industries are not as commonly targeted. This piece of data can lead to an anchoring bias, which is the tendency to rely heavily on the first piece of information we are given. While ransomware might not be as common in your industry, that does not mean it does not exist. We need to research further rather than latching onto this data to anchor our belief.

Acknowledge and act

The best way to combat these formal fallacies and biases is for the SMB and their MSP to acknowledge these beliefs and act to challenge them through proper education. Below are some of the most effective exercises we have seen SMBs and MSPs use to better educate themselves on real versus perceived ransomware risk likelihood:

  1. Threat profiling is an exercise that collects information, from vendor partners and open-source intelligence sources, to inform which threat actors are likely to target the business, using which tactics.
  2. Data flow diagrams can help you to map out your unique operating environment and see how all your systems connect together to better inform how data moves and resides within your IT environment.
  3. A risk assessment uses the threat profile information and overlays on the data flow diagram to determine where the business is most susceptible to attacker tactics.
  4. Corrective action planning is the last exercise, where you prioritize the largest gaps in protection using a threat- and risk-informed approach.

2. Being resigned to victimhood

“Large companies and enterprises get hit with ransomware all the time. As an SMB, I don’t stand a chance. I don’t have the resources they do. This is hopeless; there’s nothing I can do about it.”

Victim mentality

This past year has seen a number of companies that were supposedly “too large and well-funded to be hacked” reporting ransomware breaches. It feels like there is a constant stream of information re-enforcing the mentality that, even with a multi-million dollar security program, an SMB will not be able to effectively defend against the adverse outcomes from ransomware. This barrage of information can make them feel a loss of control and that the world is against them.

Learned helplessness

These frequent negative outcomes for “prepared” organizations are building a sense of learned helplessness, or powerlessness, within the SMB space. If a well-funded and organized company can’t stop ransomware, why should we even try?

This mentality takes a binary view on a ransomware attack, viewing it as an all-or-nothing event. In reality, there are degrees of success of a ransomware attack. The goal of becoming immune to ransomware can spark feelings of learned helplessness, but if you reframe it as minimizing the damage a successful attack will have, this allows you to regain a sense of control in what otherwise may feel like an impossible effort.

Pessimism bias

This echo chamber of successful attacks (and thus presumed unsuccessful mitigations) is driving a pessimism bias. As empathetic beings, we feel the pain of these attacked organizations as though it were our own. We then tie this negative emotion to our expectation of an event (i.e. a ransomware attack), creating the expectation of a negative outcome for our own organization.

Acknowledge and act

Biases and beliefs shape our reality. If an SMB believes they are going to fall victim to ransomware and fails to protect against it, they actually make that exact adverse outcome more likely.

Despite the fear and uncertainty, the most important variable missing from this mental math is environment complexity. The more complex the environment, the more difficult it is to protect. SMBs have an advantage over their large-business counterparts, as the SMB IT environment is usually easier to control with the right in-house tech staff and/or MSP partners. That means SMBs are better situated than large companies to deter and recover from attacks — with the right strategic investments.

Check back with us next week, when we’ll tackle the third and fourth major fallacies that hold SMBs back from securing themselves against ransomware.

Additional reading:

NEVER MISS A BLOG

Get the latest stories, expertise, and news about security today.

Migration updates announced at re:Invent 2021

Post Syndicated from Angélica Ortega original https://aws.amazon.com/blogs/architecture/migration-updates-announced-at-reinvent-2021/

re:Invent is a yearly event that offers learning and networking opportunities for the global cloud computing community. 2021 marks the launch of several new features in different areas of cloud services and migration.

In this blog, we’ll cover some of the most important recent announcements.

AWS Mainframe Modernization (Preview)

Mainframe modernization has become a necessity for many companies. One of the main drivers fueling this requirement is the need for agility, as the market constantly demands new functionalities. The mainframe platform, due to its complex dependencies, long procurement cycles, and escalating costs, makes it impossible for companies to innovate at the needed pace.

Mainframe modernization can be a complex undertaking. To assist you, we have launched a comprehensive platform, called AWS Mainframe Modernization, that enables two popular migration patterns: replatforming, and automated refactoring.

AWS Mainframe Modernization flow

Figure 1. AWS Mainframe Modernization flow

AWS Migration and Modernization Competency

Application modernization is becoming an important migration strategy, especially for strategic business applications. It brings many benefits: software licensing and operation cost optimization, better performance, agility, resilience, and more. Selecting a partner with the required expertise can help reduce the time and risk for these kinds of projects. In the next section, you’ll find a summary of the experience required by a partner to get the AWS Migration and Modernization Competency. More information can be found at AWS Migration Competency Partners.

AWS Application Migration Service (AWS MGN)

AWS MGN is recommended as the primary migration service for lift and shift migrations. Customers currently using AWS Server Migration Service are encouraged to switch to it for future migrations.

Starting in November 2021, AWS MGN supports agentless replication from VMWare vCenter versions 6.7 and 7.0 to the AWS Cloud. This new feature is intended for users who want to rehost their applications to AWS, but cannot install the AWS Replication Agent on individual servers due to company policies or technical restrictions.

AWS Elastic Disaster Recovery

Two of the pillars of the Well-Architected Framework are Operational Excellence and Reliability. Both are directly concerned with the capability of a service to recover and work efficiently. AWS Elastic Disaster Recovery is a new service to help you to minimize downtime and data loss with fast, reliable, and recovery of on-premises and cloud-based applications. It uses storage, compute, point-in-time recovery, and cost-optimization.

AWS Resilience Hub

AWS Resilience Hub is a service designed to help customers define, measure, and manage the resilience of their applications in the cloud. This service helps you define RTO (Recovery Time Objective) and RPO (Recovery Point Objective) and evaluates the configuration to meet the requirements defined. Aligned with the AWS Well-Architected Framework, this service can recover applications deployed with AWS CloudFormation, and integrates with AWS Fault Injection Simulator, AWS Systems Manager, or Amazon CloudWatch.

AWS Migration Hub Strategy Recommendations

One of the critical tasks in a migration is determining the right strategy. AWS Migration Hub can help you build a migration and modernization strategy for applications running on-premises or in AWS. AWS Migration Hub Strategy Recommendations were announced on October 2021. It’s designed to be the starting point for your cloud journey. It helps you to assess the appropriate strategy to transform your portfolios to use the full benefits of cloud services.

AWS Migration Hub Refactor Spaces (Preview)

Refactoring is the migration strategy that requires the biggest effort, but it permits you to take full advantage of cloud-native features to improve agility, performance, and scalability. AWS Migration Hub Refactor Spaces is the starting point for incremental application refactoring to microservices in AWS. It will help you reduce the undifferentiated heavy lifting of building and operating your AWS infrastructure for incremental refactoring.

AWS Database Migration Service

AWS Database Migration Service (AWS DMS) is a service that helps you migrate databases to AWS quickly and securely.

AWS DMS Fleet Advisor is a new free feature of AWS DMS that enables you to quickly build a database and analytics migration plan, by automating the discovery and analysis of your fleet. AWS DMS Fleet Advisor is intended for users looking to migrate a large number of database and analytic servers to AWS.

AWS Microservice Extractor for .NET is a new free tool and simplifies the process of re-architecting applications into smaller code projects. Modernize and transform your .NET applications with an assistive tool that analyzes source code and runtime metrics. It creates a visual representation of your application and its dependencies.

This tool visualizes your applications source code, helps with code refactoring, and assists in extraction of the code base into separate code projects.  Teams can then develop, build, and operate independently to improve agility, uptime, and scalability.

AWS Migration Evaluator

AWS Migration Evaluator (ME) is a migration assessment service that helps you create a directional business case for AWS Cloud planning and migration. Building a business case for the cloud can be a time-consuming process on your own. With Migration Evaluator, organizations can accelerate their evaluation and decision-making for migration to AWS. During 2021, there were some existing improvements to mention:

  • Quick Insights. This new capability of Migration Evaluator, provides customers with a one-page summary of their projected AWS costs, based on measured on-premises provisioning and utilization.
  • Enhanced Microsoft SQL Discovery. This is a new feature of the Migration Evaluator Collector, which assists you by including your SQL Server environment in their migration assessment.
  • Agentless Collection for Dependency Mapping. The ME Collector now enables agentless network traffic collection to be sent to the customer’s AWS Migration Hub account.

AWS Amplify Studio

This is a visual development environment that offers frontend developers new features to accelerate UI development with minimal coding, while integrating with Amplify. Read Introducing AWS Amplify Studio.

Conclusion

Migration is a crucial process for many enterprises as they move from on-premises systems to the cloud. It helps accelerate your cloud journey, and offers additional tools and methodologies created by AWS. AWS has created and is continually improving services and features to optimize the migration process and help you reach your business goals faster.

Related information

Calling All Security Researchers: Join the Backblaze Bug Bounty Program

Post Syndicated from Ola Nordstrom original https://www.backblaze.com/blog/calling-all-security-researchers-join-the-backblaze-bug-bounty-program/

Here at Backblaze, we help people build applications, host content, manage media, back up and archive data, and more securely in the cloud—and that “securely” part of the equation has always been paramount. We use a variety of tools and techniques to stay ahead of any potential security threats, including our participation over the past year plus in the Bugcrowd security platform. Today, we are opening up our Bugcrowd Bug Bounty Program to all security researchers.

Now, anyone can join Bugcrowd and start hacking away at our desktop and mobile apps, APIs, or web applications in order to help us find any vulnerabilities and strengthen the security of our services. Read on to learn more about the program and the other measures we take to spot and address potential security vulnerabilities.

Join Ola Nordstrom, Lead Application Security Engineer; Chris Vickery, Senior Risk Assessment Specialist; and Pat Patterson, Chief Developer Evangelist, on April 21, 2022 at 1 p.m. PDT to learn more about why we decided to implement the Bugcrowd Bug Bounty Program, how it fits into the Backblaze security portfolio, and how you can join in on either side: as hacker or hackee.
 
➔ Register for the Webinar Today

How Backblaze Keeps Customer Data Safe

Over the years, Backblaze has consistently invested in maintaining and upgrading its security portfolio. User files are encrypted by default, we also support server-side encryption for the Backblaze S3 Compatible API, and have doubled the size of our Security team over the last year under the leadership of CISO Mark Potter.

But all those security features and frankly all software, not just Backblaze, are vulnerable to security bugs that can expose user information and data. Oftentimes, these are caused by implementation mistakes or changes in how a piece of software is used over time. The recent Log4j (aka Log4Shell) vulnerability affected nearly everyone due to its ubiquitous use across software platforms and the industry as a whole.

I’ve been working to secure software my whole career. Before the advent of crowdsourced security platforms such as Bugcrowd, managing vulnerability reports was a painful task. Emails, typically sent to [email protected], were copied back and forth between bug tracking platforms. Reviewing submissions and gathering metrics was difficult since every engineering team or organization always had their own process for tagging and categorizing bug reports. Everything was copied back and forth to make any sense of the data (Think Excel spreadsheets!). In a world where zero-day vulnerabilities are commonplace, such processes are just too slow and you end up playing catch-up with the bad guys.

How Does Bugcrowd Fit Into the Backblaze Security Portfolio?

Bugcrowd takes the grunt work out of the process to let us focus on addressing the vulnerability and communicating with researchers. Bugcrowd encourages white hat hackers to attack businesses, find vulnerabilities in their software and processes, and aid in guiding the remediation of those vulnerabilities before they can be exploited by anyone else.

What’s more, and perhaps most important to security researchers around the world, is that Bugcrowd allows us to pay security researchers for finding vulnerabilities. Without Bugcrowd, Backblaze wouldn’t have a cost-effective way to pay for a bug report from a researcher in another country or another continent. It’s only fair we pay for the work they do to help us out, and in addition, having a public program ensures transparency and fairness for everyone.

How You Can Join the Backblaze Bugcrowd Bug Bounty Program

Backblaze’s private beta has been running for over a year, but now that the program is public, any interested security researcher can sign up to hack away the company’s in-scope products and networks. If you think you’ve found a vulnerability or you’d like more information about the in-scope products, URLs, or bounty ranges, check out the Backblaze Bugcrowd Bug Bounty Program here. And, don’t forget to register for our webinar to learn more about the program.

The post Calling All Security Researchers: Join the Backblaze Bug Bounty Program appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

[$] A way out for a.out

Post Syndicated from original https://lwn.net/Articles/888741/

The a.out executable
format dates back to the earliest days of Linux — and before. It has
not been used in any serious way for decades, but support still exists in
the Linux kernel and has resisted all attempts at its removal. Back in
January, Borislav Petkov tried yet
again to delete support for this format, leading to another extended
discussion. There is one difference this time around, though: the effort
to get rid of a.out support might just succeed.

Reflecting on Women’s History Month at Rapid7

Post Syndicated from Rapid7 original https://blog.rapid7.com/2022/03/24/reflecting-on-womens-history-month-at-rapid7/

Reflecting on Women’s History Month at Rapid7

During Women’s History Month, we invited some of our team members to share their best advice for other women in technology, celebrate their strengths, and reflect on how they’ve challenged convention within their roles and built their networks.

Reflecting on Women’s History Month at Rapid7

What is the best advice that someone has given you in your career?

Nino Nardize, Director, Technical Customer Success: One piece of advice that resonated with me early on was that you have to be comfortable making decisions with only 80% of the information available. That stuck with me because I think, as women, sometimes we feel we need to have all of the information, be 100% qualified for a role, or be able to achieve perfection. In reality, things don’t always work out that way. I’ve found it’s important to ask ourselves, “Have we done our best to gather the right information in relation to this decision?” Even if that means we still have things outstanding, that’s OK. We have to be comfortable with taking risks and feeling good about moving forward with what we have in front of us.

Jane Man, Director, Product Management – VRM: The most important advice I got early on was to be open and always look for opportunities to grow. If you are at a point in your career where you think to yourself, “I’ve got this,” and you know exactly what you are doing every day, you probably aren’t being challenged enough. This perspective has always pushed me to look for areas where I don’t feel comfortable and to seek out areas to grow my career — and myself as a person.

Jessica Reinne, Account Executive, Large: As a sales executive, the best advice I received was that buying is emotional, and people buy products most often from people they trust and have a relationship with. I’ve found firsthand that once you establish that rapport, people will want to engage with you and either communicate what they need in order to move forward or — just as valuable — tell you why they aren’t moving forward or why something isn’t actually working. When you have that open dialogue, you can really be a better partner to them, versus trying to sell them something they aren’t bought into.

Noreen Camelo, VP Enterprise Applications: One piece of advice I heard was to be brave, be brilliant, but be brief. I think this is important because sometimes we can get caught up in explaining our ideas or spend too much time setting the stage. As you progress in your career and are working with executives, it’s important that your key message comes across very clearly, as everyone is pressed for time. Be brave and speak up when you see opportunities, be brilliant and always keep challenging convention, but be brief and make sure to get your thoughts across in a succinct way.

Sarah Sidford, Manager, Commercial Sales: In life, and especially for women, it sometimes can feel like you’re trying to juggle a bunch of different balls in the air. You have your career as one ball, your travel plans as another, your family as another, your social life, and so on. It’s important to remember that most of those balls are rubber, if you drop them, they will bounce right back and be OK. But family and health are two that are glass — and you can never, ever drop them. Keeping perspective about what’s really important with a focus on family and health being most important can help you prioritize and know when you need to let go of another one of those rubber balls.

Paola Chadwell, VP, Customer Success Management: Always advocate for yourself — especially when negotiating your salary. I think, as women, we hesitate to ask for more because we’re afraid we will lose the offer that’s already on the table. In reality, the worst someone can say is no, or maybe they don’t go as high as you ask but are able to meet you in the middle somewhere. I’ve negotiated every salary throughout my career, and I think it’s helped me to own my voice and advocate for my worth.

Turning that around, what advice would you give to women who are early in their career and looking to challenge convention?

Jane Man: I would say to be confident in your own voice. At the end of the day, the goal is to create a better solution and work more effectively, and that means that when you have something to add, it’s your responsibility to speak up and you shouldn’t be afraid to do that — because it makes all of us better and stronger.

Paola Chadwell: When I was younger, I moved around a lot. The process of putting myself out there and making friends throughout childhood has translated into my ability to build strong relationships in my career. I’m in customer success, so I have to not only build strong relationships with our customers, but I also be able to have internal and cross-functional relationships so that I can be a better advocate for our customer needs and deliver the right solutions for their needs.

Nino Nardize: In the workplace, there are a number of different ways that each individual can contribute at a given time. Whether it’s in meetings or through a one-to-one conversation, each person’s ability to influence others can be a little bit different. Find where you are most comfortable and can create impact in a meaningful way and lean into that — don’t fight it to be something you are not.

How are you challenging convention in your role at Rapid7?

Sarah Sidford: I think one way that I challenge convention is that I am very vocal about the need to hire more women in sales. There are still so many double standards for women in sales — we might describe a male as being assertive and have a positive association with that behavior, but when a woman is assertive, we’re quick to judge her as being pushy. The more we can prioritize diversifying our teams, the more we can start to challenge these stereotypes. I think a lot of times, people hire people who remind them of themselves — so if we have more men in leadership positions doing the hiring, how does it hurt those efforts if they are carrying that internal bias? I’m working closely with our talent acquisition team to really change the traditional landscape of sales and prioritize bringing a more balanced workforce into the field.

Jessica Rennie: I’m someone who is really open-minded and willing to try new things. One example is that when I joined the company, I saw an opportunity and went out on a limb to create a new program. At first, it can be challenging to get something like this that is new up and running, but we were able to create this network of give and get with our prospects that has ultimately benefited our business and our sales organization. It was great to be able to formulate a vision and strategy and be supported along the way by my manager and peers.

Nino Nardize: I challenge convention by encouraging different perspectives. Diversity of thought and experiences is a crucial component to any team. I can only be one voice, and my voice is reflective of my own personal journey. Whenever we are having a crucial conversation, we need to have the representation of multiple perspectives in order to make educated decisions. Asking ourselves, “What else are we missing? Are there too many voices from one business group and not enough from another?” can lead to a better decision and product in the end.

What strengths do you believe your identity and personal experiences bring to your role?

Jane Man: I come from an immigrant family. My parents immigrated from the big city of Hong Kong to the tiny island nation of New Zealand. Through my experiences, I’ve become interested in what makes people think a certain way, and I often find myself asking questions that dig into what we are doing and the purpose or “why” behind it. That natural curiosity is something that comes from being part of that small island community.

Jessica Reinne: I didn’t come from a cybersecurity background prior to Rapid7, but I was in the startup world. Some of the places I worked earlier in my career were so new that on some days the heat didn’t even work or the lights wouldn’t turn on — so I experienced the early growth grind that happens in the technology world where you have this pressure to prove yourself and everyone’s work has such a direct impact on the company. I developed almost this chip on my shoulder where I really want to push myself and always have that grit and determination. Doing it at Rapid7, where we are so established and have not only a great team and product but great amenities and support systems, really just puts it on a whole new level.

Noreen Camelo: I bring a lot of optimism to my team, and I think some of that comes from my past roles and being able to put things into perspective. Earlier in my career I worked in Oncology, so there was a lot of pressure around our decisions because of the impact it had on patients’ lives. When we feel stressed out or a task seems too big for us to tackle, I try to help my team keep things in perspective so they can prioritize, while encouraging them to find creative solutions. I really do believe that there is no challenge we cannot solve as long as we are willing to take the right amount of time and work together.

It’s often said that an important part of being successful is building a supportive network around you. How have you built your own personal network, and how has it helped you be successful?

Paola Chadwell: I think, first off, women sometimes have a tendency to look at other women as competition rather than allies. That’s not a mindset I ever want to have or that I want to encourage people to challenge, because we all have so much to add and to contribute to one another. I’ve been so lucky to have been surrounded by such strong and powerful women at Rapid7 and at previous companies. I’ve been monitored by them and have also been able to be a mentor to them at times. The beautiful thing about building a network and having a mentor is that it becomes a two-way street. We have so much to learn from one another and can really help each other grow.

Noreen Camelo: I’ve built my network through the different roles and companies I’ve been at. It’s been a proactive experience of reaching out and staying in touch, and the result is a diverse network of people to lean on where we can all give guidance to each other at different points. What I’ve experienced is that your network is a huge part of advancing your career, as well as rounding you out as the leader you are.

Sarah Sidford: I’m grateful to have been able to create such a great network of women around  me. In sales, it’s important to have relationships where you can be honest about what you’re going through and can talk about when you are having a hard day or when things are also going well. As women, we don’t want to share our struggles and appear weak, but then at the same time, you don’t want to celebrate or you will be seen as braggadocious — so you need to have a safe space to share that vulnerability and the highs and the lows, especially when it’s people who have also been through it and who can share their experiences with you, too. Whether it’s asking about what to wear to a client onsite or working through a unique challenge with a customer, having that safe space can help you feel so much more prepared and empowered. As a leader, that’s the kind of space I want to create with my team.

Want join our team? We’re hiring! Browse our open roles at Rapid7 here.

Additional reading:

Security updates for Thursday

Post Syndicated from original https://lwn.net/Articles/889120/

Security updates have been issued by Debian (php-twig), Mageia (abcm2ps, libpano13, and pesign), openSUSE (nextcloud and xen), Oracle (kernel, kernel-container, and openssl), SUSE (java-1_7_1-ibm and xen), and Ubuntu (linux-oem-5.14, openvpn, and thunderbird).

Cloudflare Radar’s new ASN pages

Post Syndicated from Celso Martinho original https://blog.cloudflare.com/asn-on-radar/

Cloudflare Radar’s new ASN pages

Cloudflare Radar’s new ASN pages

An AS, or Autonomous System, is a group of routable IP prefixes belonging to a single entity, and is one of the key building blocks of the Internet. Internet providers, public clouds, governments, and other organizations have one or more ASes that they use to connect their users or systems to the rest of the Internet by advertising how to reach them.

Per AS traffic statistics and trends help when we need insight into unusual events, like Internet outages, infrastructure anomalies, targeted attacks, or any other changes from service providers.

Today, we are opening more of our data and launching the Cloudflare Radar pages for Autonomous Systems. When navigating to a country or region page on Cloudflare Radar you will see a list of five selected ASes for that country or region. But you shouldn’t feel limited to those, as you can deep dive into any AS by plugging its ASN (Autonomous System Number) into the Radar URL (https://radar.cloudflare.com/asn/<number>). We have excluded some statistical trends from ASes with small amounts of traffic as that data would be difficult to interpret.

Cloudflare Radar’s new ASN pages

The AS page is similar to the country page on Cloudflare Radar. You can find traffic levels, protocol use, and security details such as application and network-level DDoS attack information. Additionally, we show a geographical distribution map of the traffic and the volume of BGP announcements we see for the list of prefixes associated with the specific AS.

Cloudflare Radar’s new ASN pages

A sudden increase in BGP announcements often suggests disruptive changes to the Internet in the region or institution associated with the AS. Spikes in BGP announcements were visible when the submarine cable was cut in Tonga in 2022, on the Facebook outage in October 2021, and when governments limited the Internet access in their countries (as seen in Sudan and Syria in 2021).

Cloudflare Radar’s new ASN pages

At Cloudflare, we are committed to keep increasing transparency on the inner workings of the Internet, so that we can all do our part in keeping the Internet more open and secure for everyone. Keep an eye on Cloudflare Radar for more insights like these.

How telematics helps Grab to improve safety

Post Syndicated from Grab Tech original https://engineering.grab.com/telematics-at-grab

Telematics is a collection of sensor data such as accelerometer data, gyroscope data, and GPS data that a driver’s mobile phone provides, and we collect, during the ride. With this information, we apply data science logic to detect traffic events such as harsh braking, acceleration, cornering, and unsafe lane changes, in order to help improve our consumers’ ride experience.

Introduction

As Grab grows to meet our consumers’ needs, the number of driver-partners has also grown. This requires us to ensure that our consumers’ safety continues to remain the highest priority as we scale. We developed an in-house telematics engine which uses mobile phone sensors to determine, evaluate, and quantify the driving behaviour of our driver-partners. This telemetry data is then evaluated and gives us better insights into our driver-partners’ driving patterns.

Through our data, we hope to improve our driver-partners’ driving habits and reduce the likelihood of driving-related incidents on our platform. This telemetry data also helps us determine optimal insurance premiums for driver-partners with risky driving patterns and reward driver-partners who have better driving habits.

In addition, we also merge telematics data with spatial data to further identify areas where dangerous driving manoeuvres happen frequently. This data is used to inform our driver-partners to be alert and drive more safely in such areas.

Background

With more consumers using the Grab app, we realised that purely relying on passenger feedback is not enough; we had no definitive way to tell which driver-partners were actually driving safely, when they deviated from their routes or even if they had been involved in an accident.

To help address these issues, we developed an in-house telematics engine that analyses telemetry data, identifies driver-partners’ driving behaviour and habits, and provides safety reports for them.

Architecture details

Real time ingestion architecture

As shown in the diagram, our telematics SDK receives raw sensor data from our driver-partners’ devices and processes it in two ways:

  1. On-device processing for crash detection: Used to determine situations such as if the driver-partner has been in an accident.
  2. Raising traffic events and generating safety reports after each job: Useful for detecting events like speeding and harsh braking.

Note: Safety reports are generated by our backend service using sensor data that is only uploaded as a text file after each ride.

Implementation

Our telematics framework relies on accelerometer, gyroscope and GPS sensors within the mobile device to infer the vehicle’s driving parameters. Both accelerometer and gyroscope are triaxial sensors, and their respective measurements are in the mobile device’s frame of reference.

That being said, the data collected from these sensors have no fixed sample rate, so we need to implement sensor data time synchronisation. For example, there will be temporal misalignment between gyroscope and accelerometer data if they do not share the same timestamp. The sample rate that comes from the accelerometer and gyroscope also varies independently. Therefore, we need to uniformly sample the sensor data to be at the same frequency rate.

This synchronisation process is done in two steps:

  1. Interpolation to uniform time grid at a reasonably higher frequency.
  2. Decimation from the higher frequency to the output data rate for accelerometer and gyroscope data.

We then use the Fourier Transform to transform a signal from time domain to frequency domain for compression. These components are then written to a text file on the mobile device, compressed, and uploaded after the end of each ride.

Learnings/Conclusion

There are a few takeaways that we learned from this project:

  • Sensor data frequency: There are many device manufacturers out there for Android and each one of them has a different sensor chipset. The frequency of the sensor data may vary from device to device.
  • Four-wheel (4W) vs two-wheel (2W): The behaviour is different for a driver-partner on 2W vs 4W, so we need different rules for each.
  • Hardware axis-bias: The device may not be aligned with the vehicle during the ride. It cannot be assumed that the phone will remain in a fixed orientation throughout the trip, so the mobile device sensors might not accurately measure the acceleration/braking or sharp turning of the vehicle.
  • Sensor noise: There are artifacts in sensor readings, which are basically a single outlier event that represents an error and is not a valid sensor reading.
  • Time-synchronisation: GPS, accelerometer, and gyroscope events are captured independently by three different sensors and have different time formats. These events will need to be transformed into the same time grid in order to work together. For example, the GPS location from 30 seconds prior to the gyroscope event will not work as they are out of sync.
  • Data compression and network consumption: Longer rides will contain more telematics data.  It will result in a bigger upload size and increase in time for file compression.

What’s next?

There are a few milestones that we want to accomplish with our telematics framework in the future. However, our number one goal is to extend telematics to all bookings across Grab verticals. We are also planning to add more on-device rules and data processing for event detections to further eliminate future delays from backend communication for crash detection.

With the data from our telematics framework, we can improve our passengers’ experience and improve safety for both passengers and driver-partners.

Join us

Grab is a leading superapp in Southeast Asia, providing everyday services that matter to consumers. More than just a ride-hailing and food delivery app, Grab offers a wide range of on-demand services in the region, including mobility, food, package and grocery delivery services, mobile payments, and financial services across over 400 cities in eight countries.

Powered by technology and driven by heart, our mission is to drive Southeast Asia forward by creating economic empowerment for everyone. If this mission speaks to you, join our team today!

The collective thoughts of the interwebz