CLI tricks every developer should know

Post Syndicated from Kedasha Kerr original https://github.blog/2023-04-26-cli-tricks-every-developer-should-know/

The CLI is a critical component of a developer’s toolkit—it’s a trusty sidekick that offers flexibility and control. You can tell it what to do by simply typing in a specific command, and it will execute those commands like moving files, running programs, or even starting up a server, immediately. That being said, the CLI can be daunting to beginners, especially when you’re not sure which commands to run (and who hasn’t turned to Google to find a command they need?).

In this blog, we’ve compiled some important tricks and commands that every developer should know from GitHub’s own engineers. By mastering these basic techniques, developers can become more efficient at working with the command line and gain a deeper understanding of how the underlying operating system and programs work.

Components of the CLI

The CLI has two main components: the shell and the command. The shell is the interface that allows the user to enter commands and commands are the instructions that tell the computer what to do. Shells also provide a way to customize and extend the behavior of the CLI. With a shell, users can create their own scripts and aliases to automate tasks or simplify complex commands, and they can customize the behavior of the shell itself using configuration files. For this blog post, all of the examples are for Bash since it’s the most widely used shell. And if you’re using Windows, Windows Subsystem for Linux (WSL) is available if you’d like to use a Bash terminal.

To learn more about shells, you can check out our shell documentation here.

Keyboard shortcuts in the CLI

One of the easiest ways to improve your productivity on the command line is to learn some keyboard shortcuts. Below, you’ll find popular shortcuts that can save you time and effort on navigating and executing demands.

CTRL + C: Cancel the current command

CTRL + A: Move the cursor to the beginning of the line

CTRL + E: Move the cursor to the end of the line

CTRL + L: Clear the terminal screen

CTRL + _: Undo the last edit

CTRL-D: Exit shell session

TAB:Auto-complete commands, file names, and paths

CLI command history shortcuts

The command history allows you to quickly access and reuse previously executed commands rather than retype or search for the whole command. Here are a few to try out for yourself:

history n: Type this in the terminal to access the history

!!: Execute the last command

CTRL + R: Reverse search for a previously executed command

CTRL + P: Move to the previous command in the history

CTRL + N: Move to the next command in the history

CTRL + G: Abort search

Perform operations on multiple files with wildcards

Wildcards are characters that take the place of one or more other characters, and they’re used to increase the efficiency and flexibility of searches. They come in handy when you want to perform an operation on multiple files or directories that have similar names or patterns, and can save you a lot of time and effort by allowing you to specify patterns rather than list each individual file. There are three types of command wildcards:

?: Matches a single character. For example, if you type d?g, it will match anything that begins with a “d” and ends with an “g.”

*: Matches any number of characters. If you search s*n, it will match anything between “s” and “n” no matter how many characters are between the first and last letter.

[]: Matches only the characters enclosed within the square brackets.

Combine commands with pipes

A pipe (represented by the symbol |) connects the output of the command on the left side of the symbol to the input of the command on the right side. This allows users to create more complex and powerful commands by combining simpler commands together.

Here’s an example of how it’s used in a command:

ls -la | grep test | sort | uniq | wc -l: The final output of this command would be the number of files and directories in the current directory whose name contains the string “test.”

When you pipe all of these commands together, you are essentially:

  • Listing all files and directories in the current directory in long format
  • Searching for all lines that contain the string “test”
  • Sorting those lines alphabetically
  • Removing any duplicate lines
  • Counting the number of remaining lines

Note: grep is a useful CLI tool to search for matching patterns in a file. We’ll explore some more helpful CLI tools later on in this article.

Command substitution

Command substitution is a feature that allows you to execute a command with a different command. This helps you create more complex and dynamic commands by using the output of one command as an argument for another.

There are two different syntaxes for command substitution:

$(command) or `command`

Here’s a simple example:

$ echo "the date is $(date)": This will display the current date and time in a sentence.

Learning the CLI commands and tricks that work for you takes time and practice. But you’ll soon find that using the command line becomes second nature, and you’ll be able to accomplish more complex tasks with ease.

Now that we’ve covered some basics, you can begin to experiment with different options and parameters for your commands to see what they do. And stay curious! There are tons of great resources out there, like tldr or Explainshell, to help you learn new commands or shortcuts. And speaking of resources, let’s take a look at some helpful CLI tools to help you optimize the command line—and have some fun with it.

CLI tools to know

Command line tools are scripts, programs, or libraries created to solve problems that developers might encounter while coding. While these tools are largely focused on improving productivity, there are also CLI tools that can entertain you while you’re in the terminal (check out football-cli or spotify-tui for reference).

Here are a few CLI tools that we recommend.

grep or ack

grep is a command-line utility that allows users to search for specific patterns in text files. Users can search for a specific word, phrase, or regular expression, and output the matching lines. ack is designed to be a faster alternative to the traditional grep command. Some prefer ack over grep because it can automatically exclude files that are not typically used for programming, such as binary files or backup files.

jq

jq is a lightweight and powerful command-line JSON processor that allows developers to parse, filter, and transform JSON data using a range of query and manipulation functions. With jq, developers can easily extract specific data from large JSON files, format JSON output, and even convert between JSON and other data formats.

ImageMagick

Though ImageMagick has been around since the ‘90s, this tool remains the gold standard for all things image-related. It supports a wide range of image formats and can help simplify resizing, cropping, rotating, and applying filters and effects. One of the most powerful features of ImageMagick is its ability to automate batch processing of images, which is particularly useful for web developers and designers who need to resize or convert large numbers of images at once.

howdoi

howdoi provides answers to your programming questions directly in the terminal. This command-line tool is particularly useful since you don’t have to leave the CLI or open your web browser to get immediate access to programming knowledge.

Taskwarrior

Taskwarrior manages your to-do list right from the command line. It also includes a variety of reports and filters that allow you to view your tasks in different ways, like by project, due date, or priority.

GitHub CLI

GitHub CLI provides a streamlined and efficient way to manage GitHub projects and workflows directly from the command line. You can perform a range of tasks, like managing pull requests, viewing and responding to issues and comments, and reviewing code changes. You can also use it to access GitHub Actions and other features on GitHub without exiting your terminal.

The bottom line: ​​The CLI is a powerful tool for developers—especially if you prefer the keyboard over GUI interfaces. Learning how to use it and incorporating it into your daily workflow will help you become more productive and efficient, and getting familiar with these tips and tricks is a great way to start your CLI journey.

Additional resources for CLI

As you move further along in your developing journey, you’ll encounter different projects and pain points, which means that the commands and CLI tools you find useful might change and shift. There are so many diverse and useful CLI applications out there, and the repository awesome-cli-apps lists a bunch of excellent tools that are used and loved by developers around the world.

What’s next for CLI

We’ll be launching a technical preview of GitHub Copilot for CLI, which translates natural language prompts into terminal commands to help you find the exact command you need for the task at hand. Click here to get on the waitlist and try it out for yourself.

Законът за фалита на физическите лица – медийно неглижиран, но изключително важен. Защо чак сега и как ще ни засегне?

Post Syndicated from VassilKendov original http://kendov.com/personal_bankruptcy-2/

На 05 Април 2023 година, министрески съвет прие проект на закон за частния фалит.

ЗАКОНЪТ ЗА НЕСЪСТОЯТЕЛНОСТ НА ФИЗИЧЕСКИТЕ ЛИЦА” (ЗНФЛ) всъщност е много стара идея в България. Релано за него заговорихме официално още през 2009 година на пресконференция в БТА. Естествено и много преди това се е говорило за този закон, но под една или друга форма банките успяваха да усоетят работа по неговото въвеждане. Не защото има нещо лошо или защото така ще се нарушат драстично тяхните интереси, а защото така им беше по-комфортно да работят. Хипотетично се приемаше, че ако човек знае, че ще дължи неизплатения кредит цял живот и дори наследниците му ще наследят задължението, то той ще направи всичко възможно да го изплати.
Реалността обаче беше друга – банките в рамките на 2 до 3 години успяваха да „оглозгат” личното имущество на длъжника, а и той в този период успяваше да емигрира в чужнина или да се проспособи така, че да не могат да му вземат повече нищо. С две думи останалия период до живот, беше подарен на ЧСИ-тата наети от банките да събират вземанията, за да си играят на котка и мишка с нередовния длъжник по отношение на бъдещите му доходи.

Интервю по темата за Денят Започва в БНТ1


На практика до момента във всеки парламент се внасеше по един законопроект за ЗНФЛ, но никога не влизаше за гласуване. Такъв внесоха и патриотите и омбудсмана, дори БСП имаха такъв проект. Личното ми мнение е, че този закон винаги се е използвал за намек срещу банките, точно преди избори. Естествено не мога да го докажа, но ако проследите хронологията на обявените за публично обсъждане варинти на ЗНФЛ ще видите, че тезата не е съвсем лишена от логика.

През миналата 2022 година обаче приемането на такъв закон беше поставено като изискване за отпускането на България на следващия транш от стредствата по плана за възстановяване и развитие на ЕС. Когато „замирише” на пари в българския политичски живот, нещата стават на живот и смърт. Министреството на Крум Зарков (БСП) изключително бързо състави такъв закон и го публикува за обсъждане в края на 2022.
Да си призная честно аз повече забележки към закон не бях виждал. Нямаше нито една заинтересована страна, която да не го разкритикува. Най свирепи бяха атаките от асоциацията на банките, камарата на ЧСИ и знайни и незнайни асоциации.

Ето какво казват съдиите
„Съдиите имат и доста забележки по правната логика на закона. „Производството изкуствено и излишно е усложнено с привнасяне по механичен начин на норми на Търговския закон към една материя, която трябва да е проста и ясна и хората, дори сами четейки закона, да могат да потърсят защита. Законът в този му вид ще затрудни и правоспособни юристи“, каза Комитова.”

И може би с право. Аз също не намирах здрава логика в този нов закон. Сред вносителите му не видях нито един от дългогодишните поддържници на идеята за въвеждането му. Стигнах до извода, че този закон е превеждан от друга правна система. Познавайки англосаксонската, мисля, че е или от законодателството на САЩ или това на Обединеното кралство. Ако приемем тази теза, уравнението ще се  равни. В този случай нямат нужда от локални експерти и съответно предложението ще е тотално непригодно за нашето законодателство, което на свой ред се доказа с множеството отрицателни становища на всички заинтересовани страни.

Давам пример веднага:
В България дори незапознатите знаят, че ЧСИ имат изключително големи правомощия при събирането на средствата. Като цяло, ЧСИ спяват да съберат около 1.3 млр на година, което никак не е зле. Процедурите са ясни, а явно и достатъчно ефективни.
Вместо да използваме тях и изградената система за събиране на средства от длъжници, с новия закон Крум Зарков се опитва да вуведе нова фигура за нашето законодателство – „Ликвидатор на имущество на физически лица – ФЛ”. Сами разбирате, че личните финанси на един човек не са като тези на някоя корпорация. Няма толкова договори и интереси, че да се изисква специален ликвидатор работещ с години по продацбата на няколко имота и автомобила, каквото е имуществото на 99% от хората в България.
Естествено винаги може да се намери оправдание, че ще „изтървем” този 1%, но ако бяха толква загрижени вносителите, да бяха предложили да има ликвидатор само за имущества над 5млн примерно. Все пак на този ликвидатор се плаща и тогава задълежите нараства драстично. Дори схемата за заплащането му е предвидена в закона.

Далеч съм от мисълта да анализирам всички недостатъчци на този законопроект, но още в мотивите на вносителите ще видите несъотвествие. Според тях с този законопроект се цели да се съберат по-ефективно парите на кредиторите, а аз бих казал, че целта трябва да бъде да се помогне на длъжника да влезе отново в легалната икономика, да не бяга в чужбина и да се създадат условия този човек отново да плаща данъците си тук. Да отгледжа децата си и да потребява тук, а не в чужбина. Нищо подобно не се вижда като идея в този законопроет.

За  срещa с мен моля използвайте посочената форма.

[contact-form-7]

Както обичам да казвам – това е един мъртво роден закон, който в този си вид ще създаде повече проблеми, отколкото ще реши. Всичко около него се върти около определението за „добросъвестен длъжник”. Мога да си представя как се доказва „добросъвестност” в нашето законодателство.

На практика е излишно да влизам в допълнителни обяснения какво не е наред с този ЗНФЛ. Както казах още мотивите му са несъстоятелни и грешни, но ака е когато се прави набързо от незаинтересовани и видимо некомпетентни хора. В случая мнението ми съвпада с това на камарата на ЧСИ (което е доста необичайно), според които законопроекта е толкова лош, че дори не става за основа за дебати.

И така стигаме до ситуацията в която ако не го приемем, няма да получим парите от Европа, а ако пък го приемем, ще страдаме години наред. И аз самия не зная какъв е изхода от това положение, но все ми се струва, че за всички нас е по-изгодно да отложим приемането му макар и с риск да се забавят парите за възстановяване и развитие по програмата на ЕС. Д ане стане така, че после да ни трябват пари за „възстановяване от последиците от приетите набързо закони”

За съжаление лично аз не виждам друга надежда освен нови избори и нов кабинет с по-различна визия по този въпрос. В противен случай ни чакат години на хаос. Странно е, но другата надежда е банковото лоби и лобито на ЧСИ. Надявам се те да повлияят в каквато и да е посока, различна от сегашната.

Васил Кендов – финансов съветник
Лошото момче на българските финанси

The post Законът за фалита на физическите лица – медийно неглижиран, но изключително важен. Защо чак сега и как ще ни засегне? appeared first on Kendov.com.

Вторият Международен младежки литературен фестивал „Приятелството – смисъл и спасение” събира в Бургас млади автори от 9 държави

Post Syndicated from Биволъ original https://bivol.bg/vtoriqt-mezhdunaroden-festival.html

сряда 26 април 2023


Международният младежки литературен фестивал „Приятелството – смисъл и спасение” ще събере в Бургас от 19 до 21 юни 2023 година млади автори от България и чужбина и ще постави финалните…

Let’s Architect! Getting started with containers

Post Syndicated from Luca Mezzalira original https://aws.amazon.com/blogs/architecture/lets-architect-getting-started-with-containers/

Most of AWS customers building cloud-native applications or modernizing applications choose containers to run their microservices applications to accelerate innovation and time to market while lowering their total cost of ownership (TCO). Using containers in AWS comes with other benefits, such as increased portability, scalability, and flexibility.

The combination of containers technologies and AWS services also provides features such as load balancing, auto scaling, and service discovery, making it easier to deploy and manage applications at scale.

In this edition of Let’s Architect! we share useful resources to help you to get started with containers on AWS.

Container Build Lens

This whitepaper describes the Container Build Lens for the AWS Well-Architected Framework. It helps customers review and improve their cloud-based architectures and better understand the business impact of their design decisions. The document describes general design principles for containers, as well as specific best practices and implementation guidance using the Six Pillars of the Well-Architected Framework.

Take me to explore the Containers Build Lens!

Follow Containers Build Lens Best practices to architect your containers-based workloads

Follow Containers Build Lens Best practices to architect your containers-based workloads.

EKS Workshop

The EKS Workshop is a useful resource to familiarize yourself with Amazon Elastic Kubernetes Service (Amazon EKS) by practicing on real use-cases. It is built to help users learn about Amazon EKS features and integrations with popular open-source projects. The workshop is abstracted into high-level learning modules, including Networking, Security, DevOps Automation, and more. These are further broken down into standalone labs focusing on a particular feature, tool, or use case.

Once you’re done experimenting with EKS Workshop, start building your environments with Amazon EKS Blueprints, a collection of Infrastructure as Code (IaC) modules that helps you configure and deploy consistent, batteries-included Amazon EKS clusters across accounts and regions following AWS best practices. Amazon EKS Blueprints are available in both Terraform and CDK.

Take me to this workshop!

The workshop is abstracted into high-level learning modules, including Networking, Security, DevOps Automation, and more.

The workshop is abstracted into high-level learning modules, including Networking, Security, DevOps Automation, and more.

Architecting for resiliency on AWS App Runner

Learn how to architect an highly available and resilient application using AWS App Runner. With App Runner, you can start with just the source code of your application or a container image. The complexity of running containerized applications is abstracted away, including the cloud resources needed for running your web application or API. App Runner manages load balancers, TLS certificates, auto scaling, logs, metrics, teachability and more, so you can focus on implementing your business logic in a highly scalable and elastic environment.

Take me to this blog post!

A high-level architecture for an available and resilient application with AWS App Runner.

A high-level architecture for an available and resilient application with AWS App Runner

Securing Kubernetes: How to address Kubernetes attack vectors

As part of designing any modern system on AWS, it is necessary to think about the security implications and what can affect your security posture. This session introduces the fundamentals of the Kubernetes architecture and common attack vectors. It also includes security controls provided by Amazon EKS and suggestions on how to address them. With these strategies, you can learn how to reduce risk for your Kubernetes-based workloads.

Take me to this video!

Some common attack vectors that need addressing with Kubernetes

Some common attack vectors that need addressing with Kubernetes

See you next time!

Thanks for exploring architecture tools and resources with us!

Next time we’ll talk about serverless.

To find all the posts from this series, check out the Let’s Architect! page of the AWS Architecture Blog.

GCC 13.1 released

Post Syndicated from original https://lwn.net/Articles/930259/

Version
13.1 of the GCC compiler suite has been released.

This release integrates a frontend for the Modula-2 language which
was previously available separately and lays foundation for a
frontend for the Rust language which will be available in a future
release.

Other changes include the removal of support for the STABS
debugging-information format, addition of a number of C++23 features, a
number of static-analyzer improvements, support for a number of recent CPU
features, and more. See this page for details.

Security updates for Wednesday

Post Syndicated from original https://lwn.net/Articles/930258/

Security updates have been issued by Fedora (chromium, lilypond, and lilypond-doc), Oracle (java-1.8.0-openjdk), Red Hat (emacs, java-1.8.0-openjdk, java-11-openjdk, java-17-openjdk, kernel, kernel-rt, pesign, and virt:rhel, virt-devel:rhel), Scientific Linux (java-1.8.0-openjdk and java-11-openjdk), Slackware (git), SUSE (fwupd, git, helm, and runc), and Ubuntu (firefox, golang-1.18, linux-hwe-5.15, and openssl, openssl1.0).

AI to Aid Democracy

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/ai-to-aid-democracy.html

There’s good reason to fear that AI systems like ChatGPT and GPT4 will harm democracy. Public debate may be overwhelmed by industrial quantities of autogenerated argument. People might fall down political rabbit holes, taken in by superficially convincing bullshit, or obsessed by folies à deux relationships with machine personalities that don’t really exist.

These risks may be the fallout of a world where businesses deploy poorly tested AI systems in a battle for market share, each hoping to establish a monopoly.

But dystopia isn’t the only possible future. AI could advance the public good, not private profit, and bolster democracy instead of undermining it. That would require an AI not under the control of a large tech monopoly, but rather developed by government and available to all citizens. This public option is within reach if we want it.

An AI built for public benefit could be tailor-made for those use cases where technology can best help democracy. It could plausibly educate citizens, help them deliberate together, summarize what they think, and find possible common ground. Politicians might use large language models, or LLMs, like GPT4 to better understand what their citizens want.

Today, state-of-the-art AI systems are controlled by multibillion-dollar tech companies: Google, Meta, and OpenAI in connection with Microsoft. These companies get to decide how we engage with their AIs and what sort of access we have. They can steer and shape those AIs to conform to their corporate interests. That isn’t the world we want. Instead, we want AI options that are both public goods and directed toward public good.

We know that existing LLMs are trained on material gathered from the internet, which can reflect racist bias and hate. Companies attempt to filter these data sets, fine-tune LLMs, and tweak their outputs to remove bias and toxicity. But leaked emails and conversations suggest that they are rushing half-baked products to market in a race to establish their own monopoly.

These companies make decisions with huge consequences for democracy, but little democratic oversight. We don’t hear about political trade-offs they are making. Do LLM-powered chatbots and search engines favor some viewpoints over others? Do they skirt controversial topics completely? Currently, we have to trust companies to tell us the truth about the trade-offs they face.

A public option LLM would provide a vital independent source of information and a testing ground for technological choices with big democratic consequences. This could work much like public option health care plans, which increase access to health services while also providing more transparency into operations in the sector and putting productive pressure on the pricing and features of private products. It would also allow us to figure out the limits of LLMs and direct their applications with those in mind.

We know that LLMs often “hallucinate,” inferring facts that aren’t real. It isn’t clear whether this is an unavoidable flaw of how they work, or whether it can be corrected for. Democracy could be undermined if citizens trust technologies that just make stuff up at random, and the companies trying to sell these technologies can’t be trusted to admit their flaws.

But a public option AI could do more than check technology companies’ honesty. It could test new applications that could support democracy rather than undermining it.

Most obviously, LLMs could help us formulate and express our perspectives and policy positions, making political arguments more cogent and informed, whether in social media, letters to the editor, or comments to rule-making agencies in response to policy proposals. By this we don’t mean that AI will replace humans in the political debate, only that they can help us express ourselves. If you’ve ever used a Hallmark greeting card or signed a petition, you’ve already demonstrated that you’re OK with accepting help to articulate your personal sentiments or political beliefs. AI will make it easier to generate first drafts, and provide editing help and suggest alternative phrasings. How these AI uses are perceived will change over time, and there is still much room for improvement in LLMs—but their assistive power is real. People are already testing and speculating on their potential for speechwriting, lobbying, and campaign messaging. Highly influential people often rely on professional speechwriters and staff to help develop their thoughts, and AI could serve a similar role for everyday citizens.

If the hallucination problem can be solved, LLMs could also become explainers and educators. Imagine citizens being able to query an LLM that has expert-level knowledge of a policy issue, or that has command of the positions of a particular candidate or party. Instead of having to parse bland and evasive statements calibrated for a mass audience, individual citizens could gain real political understanding through question-and-answer sessions with LLMs that could be unfailingly available and endlessly patient in ways that no human could ever be.

Finally, and most ambitiously, AI could help facilitate radical democracy at scale. As Carnegie Mellon professor of statistics Cosma Shalizi has observed, we delegate decisions to elected politicians in part because we don’t have time to deliberate on every issue. But AI could manage massive political conversations in chat rooms, on social networking sites, and elsewhere: identifying common positions and summarizing them, surfacing unusual arguments that seem compelling to those who have heard them, and keeping attacks and insults to a minimum.

AI chatbots could run national electronic town hall meetings and automatically summarize the perspectives of diverse participants. This type of AI-moderated civic debate could also be a dynamic alternative to opinion polling. Politicians turn to opinion surveys to capture snapshots of popular opinion because they can only hear directly from a small number of voters, but want to understand where voters agree or disagree.

Looking further into the future, these technologies could help groups reach consensus and make decisions. Early experiments by the AI company DeepMind suggest that LLMs can build bridges between people who disagree, helping bring them to consensus. Science fiction writer Ruthanna Emrys, in her remarkable novel A Half-Built Garden, imagines how AI might help people have better conversations and make better decisions—rather than taking advantage of these biases to maximize profits.

This future requires an AI public option. Building one, through a government-directed model development and deployment program, would require a lot of effort—and the greatest challenges in developing public AI systems would be political.

Some technological tools are already publicly available. In fairness, tech giants like Google and Meta have made many of their latest and greatest AI tools freely available for years, in cooperation with the academic community. Although OpenAI has not made the source code and trained features of its latest models public, competitors such as Hugging Face have done so for similar systems.

While state-of-the-art LLMs achieve spectacular results, they do so using techniques that are mostly well known and widely used throughout the industry. OpenAI has only revealed limited details of how it trained its latest model, but its major advance over its earlier ChatGPT model is no secret: a multi-modal training process that accepts both image and textual inputs.

Financially, the largest-scale LLMs being trained today cost hundreds of millions of dollars. That’s beyond ordinary people’s reach, but it’s a pittance compared to U.S. federal military spending—and a great bargain for the potential return. While we may not want to expand the scope of existing agencies to accommodate this task, we have our choice of government labs, like the National Institute of Standards and Technology, the Lawrence Livermore National Laboratory, and other Department of Energy labs, as well as universities and nonprofits, with the AI expertise and capability to oversee this effort.

Instead of releasing half-finished AI systems for the public to test, we need to make sure that they are robust before they’re released—and that they strengthen democracy rather than undermine it. The key advance that made recent AI chatbot models dramatically more useful was feedback from real people. Companies employ teams to interact with early versions of their software to teach them which outputs are useful and which are not. These paid users train the models to align to corporate interests, with applications like web search (integrating commercial advertisements) and business productivity assistive software in mind.

To build assistive AI for democracy, we would need to capture human feedback for specific democratic use cases, such as moderating a polarized policy discussion, explaining the nuance of a legal proposal, or articulating one’s perspective within a larger debate. This gives us a path to “align” LLMs with our democratic values: by having models generate answers to questions, make mistakes, and learn from the responses of human users, without having these mistakes damage users and the public arena.

Capturing that kind of user interaction and feedback within a political environment suspicious of both AI and technology generally will be challenging. It’s easy to imagine the same politicians who rail against the untrustworthiness of companies like Meta getting far more riled up by the idea of government having a role in technology development.

As Karl Popper, the great theorist of the open society, argued, we shouldn’t try to solve complex problems with grand hubristic plans. Instead, we should apply AI through piecemeal democratic engineering, carefully determining what works and what does not. The best way forward is to start small, applying these technologies to local decisions with more constrained stakeholder groups and smaller impacts.

The next generation of AI experimentation should happen in the laboratories of democracy: states and municipalities. Online town halls to discuss local participatory budgeting proposals could be an easy first step. Commercially available and open-source LLMs could bootstrap this process and build momentum toward federal investment in a public AI option.

Even with these approaches, building and fielding a democratic AI option will be messy and hard. But the alternative—shrugging our shoulders as a fight for commercial AI domination undermines democratic politics—will be much messier and much worse.

This essay was written with Henry Farrell and Nathan Sanders, and previously appeared on Slate.com.

EDITED TO ADD: Linux Weekly News discussion.

How Dafiti made Amazon QuickSight its primary data visualization tool

Post Syndicated from Valdiney Gomes original https://aws.amazon.com/blogs/big-data/how-dafiti-made-amazon-quicksight-its-primary-data-visualization-tool/

This is a guest post by Valdiney Gomes, Hélio Leal, and Flávia Lima from Dafiti.

Data and its various uses is increasingly evident in companies, and each professional has their preferences about which technologies to use to visualize data, which isn’t necessarily in line with the technological needs and infrastructure of a company. At Dafiti, a Brazilian fashion and style e-commerce retailer, it was no different. Five tools were used by different sectors of the company, which caused misalignment and management overhead, spreading our resources thin to support them. Looking for a tool that would enable us to democratize our data, we chose Amazon QuickSight, a cloud-native, serverless business intelligence (BI) service that powers interactive dashboards that lets us make better data-driven decisions, as a corporate solution for data visualization.

In this post, we discuss why we chose QuickSight and how we implemented it.

Why we chose QuickSight

We had specific requirements for our BI solution and looked at many different options. The following factors guided our decision:

  • Tool close to data – It was important to have the data visualization tool as close to the data as possible. At Dafiti, the entire infrastructure is on AWS, and we use Amazon Redshift as our Data Warehouse. QuickSight, when using SPICE (Super-fast, Parallel, In-memory Calculation Engine), extracts data from Amazon Redshift as efficiently as possible using UNLOAD, which optimizes the use of Amazon Redshift.
  • Highly available and accessible solution – We wanted to be able to be access the tool by web or mobile interface, in addition to being able to do almost anything through API calls.
  • Serverless solution – All the other data visualization solutions that were used at Dafiti were on premises, which created unnecessary cost and effort to maintain these services, taking the focus away from what was most important to us: data.
  • Flexible pricing model – We needed a pricing model that would allow us to provide access to everyone in the company and at a price defined by usage and not by license. Thanks to AWS pay-as-you-go pricing, with more than double the number of users we had on our previous main data visualization solution, our cost with QuickSight is about 10 times lower.
  • Robust documentation – The material provided by AWS proved to be helpful, allowing our team to put the project into production.

Unifying our solution

We were previously using Qlikview, Sisense, Tableau, SAP, and Excel to analyze our data across different teams. We were already using other AWS services and learning about QuickSight when we hosted a Data Battle with AWS, a hybrid event for more than 230 Dafiti employees. This event had a hands-on approach with a workshop followed by a friendly QuickSight competition. Participants had to get information in their own dashboard to answer correctly. This 5-hour event flew by, accelerated the learning path of technical and business teams, and proved that QuickSight was the right tool for us.

QuickSight has brought all of our teams into one tool, while lowering costs by 80% and enabling us to do so much more together. Currently, over 400 employees, including our CEO, across nine different business units are using QuickSight as their sole source of truth on a daily basis. This includes human resources, auditing, and customer service, which previously had their analyses spread across several sources.

Data democratization

Data democratization is one of Dafiti’s main objectives. We believe that allowing everyone to analyze the data, following Brazilian, Argentinean, and Colombian privacy laws, unlocks potential for improving decision-making processes by extracting value from the data generated by the company. However, the democratization of data comes with the responsible use of resources. Yes, we want all users to be able to access and extract value from the data, but the cost can never be greater than the value that this generates.

How we organized the project

Data democratization drives Dafiti’s strategy. When implementing QuickSight, the obsession of becoming an even more data-driven company (we talk about this at the AWS Summit SP 2022) and having data increasingly accessible was what guided the project.

We organized QuickSight by folders, as can be seen in the following figure, and each folder represents a business area. This makes it easier to grant access and ensures that all people from the same area have access to exactly the same set of data and reports.

model of Dafiti's QuickSight folders

In this model, people from the corporate data area can view and edit any resource from any area, while customer service users can view and edit resources only for customer service.

Expanding the model a bit, the reports created by one area can be shared with others, as can be seen in the following figure, in which the SAC report was shared with Support, creating what we call a reporting portfolio.

an expansion of the folders

In this way, all users who join any of the groups will have exactly the same view as any of their peers, eliminating privileges in accessing data. In addition, the portfolio is enriched every day with reports that are created and maintained by other areas, but which may be of interest to areas other than the one responsible for creating it.

For this to work correctly, a certain rigidity is necessary in relation to the few naming and documentation standards that have been defined. On the other hand, designers have complete freedom to define the characteristics of their reports.

Another highlight in this model is that no report can be shared directly with a specific user; this restriction was defined using custom permissions in QuickSight. Therefore, the reports are always shared only through the folders. After all, we want the data to be accessible equally to everyone in the company.

Technical configurations

QuickSight offers a comprehensive API, and all the activities we carry out on a daily basis take place through these APIs. Among these activities, we highlight the granting of access and the monitoring of various aspects of the tool.

The QuickSight visual interface allows most of the tool’s maintenance activities to be performed and integration with Active Directory or the use of AWS Identity and Access Management (IAM) users is possible, but we understand that it wouldn’t be the ideal choice to grant access. Therefore, we defined an access grant flow for users and groups based on the QuickSight API, as can be seen in the following figure. In this model, the creation and removal of users is done through a JSON file with the following structure:

{
 "Version":"1.0.0",
 "Namespace":"default",
 "AwsAccountId":"<AwsAccountId>",
 "AwsRegion":"<AwsRegion>",
 "Permission":{
  "GroupList":[
   {"GroupName":"QUICKSIGHT_DATA_EDITOR"},
   {"GroupName":"QUICKSIGHT_DATA_VIEWER"},
   {"GroupName":"QUICKSIGHT_DATA_DESIGNER"},
   {"GroupName":"QUICKSIGHT_SAC_VIEWER"},
   {"GroupName":"QUICKSIGHT_SAC_DESIGNER"},
    ...
  ],
  "UserList":[
   {"UserName":"[email protected]","Active":"True","GroupList":[{"GroupName":"QUICKSIGHT_DATA_EDITOR"}]},
   {"UserName":"[email protected]","Active":"True","GroupList":[{"GroupName":"QUICKSIGHT_SAC_VIEWER"}]},
   ...
  ]
 }
}

Whenever a user needs to be added or changed, the file is edited and a pull request is submitted to GitHub. If the request is approved, an action is triggered to send the file to an Amazon Simple Storage Service (Amazon S3) bucket. From this, an AWS Lambda function is triggered that performs two activities: the first is the maintenance of users and groups, and the second is the sending of an invitation through Amazon Simple Email Service (Amazon SES) for users to join QuickSight. In our case, we opted for a personalized invitation model that would emphasize the data democratization initiative that is being conducted.

an architecture diagram from JSON to QuickSight

To monitor the tool, we implemented the architecture shown in the following figure, in which we used AWS CloudTrail to pull out the QuickSight logs and the QuickSight API to extract information from the tool’s resources, such as reports, users, datasets, data sources, and more. All of this data is processed by Glove, our data integration tool, stored in Amazon Redshift, and analyzed in QuickSight itself. This allows us to understand the behavior of our users and concentrate efforts on the most-used resources, in addition to allowing optimal cost control and the use of SPICE.

an architecture diagram from QuickSight to Redshift

To update the datasets, we don’t use the QuickSight internal scheduler, due to the large volume of data and the complexity of the DAGs. We prefer updating the datasets within our ETL (extract, transform, and load) and ELT process orchestration flow. For this purpose, we use Hanger, our orchestration tool. This approach allows the datasets to be updated only when the data source is changed and the data quality processes are executed. This model is represented by the following figure.

an architecture diagram with Redshift, Hanger, and QuickSight API

Conclusion

Choosing a data visualization tool is not a simple task. It involves many considerations, and several aspects must be analyzed in order for the choice to fit the characteristics of the company and to be consistent with the profile of business users.

For Dafiti, QuickSight was a natural choice from the moment we learned about its features. We needed a service that was in the same cloud as our main data sources, extremely fast using SPICE, and solved the maintenance and cost problem of on-premises applications. In terms of functionalities that are necessary for our business, it met our needs perfectly.

Do you want to know more about what we are doing in the data area here at Dafiti? Check out the following videos:

About the Authors

Valdiney Gomes is Data Engineering Coordinator at Dafiti. He worked for many years in software engineering, migrated to data engineering, and currently leads an amazing team responsible for the data platform for Dafiti in Latin America.

Hélio Leal is a Data Engineering Specialist at Dafiti, responsible for maintaining and evolving the entire data platform at Dafiti using AWS solutions.

Flávia Lima is a Data Engineer at Dafiti, responsible for sustaining the data platform and providing the data from many sources to internal customers.

Cross-account integration between SaaS platforms using Amazon AppFlow

Post Syndicated from Ramakant Joshi original https://aws.amazon.com/blogs/big-data/cross-account-integration-between-saas-platforms-using-amazon-appflow/

Implementing an effective data sharing strategy that satisfies compliance and regulatory requirements is complex. Customers often need to share data between disparate software as a service (SaaS) platforms within their organization or across organizations. On many occasions, they need to apply business logic to the data received from the source SaaS platform before pushing it to the target SaaS platform.

Let’s take an example. AnyCompany’s marketing team hosted an event at the Anaheim Convention Center, CA. The marketing team created leads based on the event in Adobe Marketo. An automated process downloaded the leads from Marketo in the marketing AWS account. These leads are then pushed to the sales AWS account. A business process picks up those leads, filters them based on a “Do Not Call” criteria, and creates entries in the Salesforce system. Now, the sales team can pursue those leads and continue to track the opportunities in Salesforce.

In this post, we show how to share your data across SaaS platforms in a cross-account structure using fully managed, low-code AWS services such as Amazon AppFlow, Amazon EventBridge, AWS Step Functions, and AWS Glue.

Solution overview

Considering our example of AnyCompany, let’s look at the data flow. AnyCompany’s Marketo instance is integrated with the producer AWS account. As the leads from Marketo land in the producer AWS account, they’re pushed to the consumer AWS account, which is integrated to Salesforce. Business logic is applied to the leads data in the consumer AWS account, and then the curated data is loaded into Salesforce.

We have used a serverless architecture to implement this use case. The following AWS services are used for data ingestion, processing, and load:

  • Amazon AppFlow is a fully managed integration service that enables you to securely transfer data between SaaS applications like Salesforce, SAP, Marketo, Slack, and ServiceNow, and AWS services like Amazon S3 and Amazon Redshift, in just a few clicks. With AppFlow, you can run data flows at nearly any scale at the frequency you choose—on a schedule, in response to a business event, or on demand. You can configure data transformation capabilities like filtering and validation to generate rich, ready-to-use data as part of the flow itself, without additional steps. Amazon AppFlow is used to download leads data from Marketo and upload the curated leads data into Salesforce.
  • Amazon EventBridge is a serverless event bus that lets you receive, filter, transform, route, and deliver events. EventBridge is used to track the events like receiving the leads data in the producer or consumer AWS accounts and then triggering a workflow.
  • AWS Step Functions is a visual workflow service that helps developers use AWS services to build distributed applications, automate processes, orchestrate microservices, and create data and machine learning (ML) pipelines. Step Functions is used to orchestrate the data processing.
  • AWS Glue is a serverless data preparation service that makes it easy to run extract, transform, and load (ETL) jobs. An AWS Glue job encapsulates a script that reads, processes, and then writes data to a new schema. This solution uses Python 3.6 AWS Glue jobs for data filtration and processing.
  • Amazon Simple Storage Service (Amazon S3) is an object storage service offering industry-leading scalability, data availability, security, and performance. Amazon S3 is used to store the leads data.

Let’s review the architecture in detail. The following diagram shows a visual representation of how this integration works.

The following steps outline the process for transferring and processing leads data using Amazon AppFlow, Amazon S3, EventBridge, Step Functions, AWS Glue, and Salesforce:

  1. Amazon AppFlow runs on a daily schedule and retrieves any new leads created within the last 24 hours (incremental changes) from Marketo.
  2. The leads are saved as Parquet format files in an S3 bucket in the producer account.
  3. When the daily flow is complete, Amazon AppFlow emits events to EventBridge.
  4. EventBridge triggers Step Functions.
  5. Step Functions copies the Parquet format files containing the leads from the producer account’s S3 bucket to the consumer account’s S3 bucket.
  6. Upon a successful file transfer, Step Functions publishes an event in the consumer account’s EventBridge.
  7. An EventBridge rule intercepts this event and triggers Step Functions in the consumer account.
  8. Step Functions calls an AWS Glue crawler, which scans the leads Parquet files and creates a table in the AWS Glue Data Catalog.
  9. The AWS Glue job is called, which selects records with the Do Not Call field set to false from the leads files, and creates a new set of curated Parquet files. We have used an AWS Glue job for the ETL pipeline to showcase how you can use purpose-built analytics service for complex ETL needs. However, for simple filtering requirements like Do Not Call, you can use the existing filtering feature of Amazon AppFlow.
  10. Step Functions then calls Amazon AppFlow.
  11. Finally, Amazon AppFlow populates the Salesforce leads based on the data in the curated Parquet files.

We have provided artifacts in this post to deploy the AWS services in your account and try out the solution.

Prerequisites

To follow the deployment walkthrough, you need two AWS accounts, one for the producer and other for the consumer. Use us-east-1 or us-west-2 as your AWS Region.

Consumer account setup:

Stage the data

To prepare the data, complete the following steps:

  1. Download the zipped archive file to use for this solution and unzip the files locally.

The AWS Glue job uses the glue-job.py script to perform ETL and populates the curated table in the Data Catalog.

  1. Create an S3 bucket called consumer-configbucket-<ACCOUNT_ID> via the Amazon S3 console in the consumer account, where ACCOUNT_ID is your AWS account ID.
  2. Upload the script to this location.

Create a connection to Salesforce

Follow the connection setup steps outlined in here. Please make a note of the Salesforce connector name.

Create a connection to Salesforce in the consumer account

Follow the connection setup steps outlined in Create Opportunity Object Flow.

Set up resources with AWS CloudFormation

We provided two AWS CloudFormation templates to create resources: one for the producer account, and one for the consumer account.

Amazon S3 now applies server-side encryption with Amazon S3 managed keys (SSE-S3) as the base level of encryption for every bucket in Amazon S3. Starting January 5, 2023, all new object uploads to Amazon S3 are automatically encrypted at no additional cost and with no impact on performance. We use this default encryption for both producer and consumer S3 buckets. If you choose to bring your own keys with AWS Key Management Service (AWS KMS), we recommend referring to Replicating objects created with server-side encryption (SSE-C, SSE-S3, SSE-KMS) for cross-account replication.

Launch the CloudFormation stack in the consumer account

Let’s start with creating resources in the consumer account. There are a few dependencies on the consumer account resources from the producer account. To launch the CloudFormation stack in the consumer account, complete the following steps:

  1. Sign in to the consumer account’s AWS CloudFormation console in the target Region.
  2. Choose Launch Stack.
    BDB-2063-launch-cloudformation-stack
  3. Choose Next.
  4. For Stack name, enter a stack name, such as stack-appflow-consumer.
  5. Enter the parameters for the connector name, object, and producer (source) account ID.
  6. Choose Next.
  7. On the next page, choose Next.
  8. Review the details on the final page and select I acknowledge that AWS CloudFormation might create IAM resources.
  9. Choose Create stack.

Stack creation takes approximately 5 minutes to complete. It will create the following resources. You can find them on the Outputs tab of the CloudFormation stack.

  • ConsumerS3Bucket – consumer-databucket-<consumer account id>
  • Consumer S3 Target Folder – marketo-leads-source
  • ConsumerEventBusArn – arn:aws:events:<region>:<consumer account id>:event-bus/consumer-custom-event-bus
  • ConsumerEventRuleArn – arn:aws:events:<region>:<consumer account id>:rule/consumer-custom-event-bus/consumer-custom-event-bus-rule
  • ConsumerStepFunction – arn:aws:states:<region>:<consumer account id>:stateMachine:consumer-state-machine
  • ConsumerGlueCrawler – consumer-glue-crawler
  • ConsumerGlueJob – consumer-glue-job
  • ConsumerGlueDatabase – consumer-glue-database
  • ConsumerAppFlow – arn:aws:appflow:<region>:<consumer account id>:flow/consumer-appflow

Producer account setup:

Create a connection to Marketo

Follow the connection setup steps outlined in here. Please make a note of the Marketo connector name.

Launch the CloudFormation stack in the producer account

Now let’s create resources in the producer account. Complete the following steps:

  1. Sign in to the producer account’s AWS CloudFormation console in the source Region.
  2. Choose Launch Stack.
    BDB-2063-launch-cloudformation-stack
  3. Choose Next.
  4. For Stack name, enter a stack name, such as stack-appflow-producer.
  5. Enter the following parameters and leave the rest as default:
    • AppFlowMarketoConnectorName: name of the Marketo connector, created above
    • ConsumerAccountBucket: consumer-databucket-<consumer account id>
    • ConsumerAccountBucketTargetFolder: marketo-leads-source
    • ConsumerAccountEventBusArn: arn:aws:events:<region>:<consumer account id>:event-bus/consumer-custom-event-bus
    • DefaultEventBusArn: arn:aws:events:<region>:<producer account id>:event-bus/default


  6. Choose Next.
  7. On the next page, choose Next.
  8. Review the details on the final page and select I acknowledge that AWS CloudFormation might create IAM resources.
  9. Choose Create stack.

Stack creation takes approximately 5 minutes to complete. It will create the following resources. You can find them on the Outputs tab of the CloudFormation stack.

  • Producer AppFlow – producer-flow
  • Producer Bucket – arn:aws:s3:::producer-bucket.<region>.<producer account id>
  • Producer Flow Completion Rule – arn:aws:events:<region>:<producer account id>:rule/producer-appflow-completion-event
  • Producer Step Function – arn:aws:states:<region>:<producer account id>:stateMachine:ProducerStateMachine-xxxx
  • Producer Step Function Role – arn:aws:iam::<producer account id>:role/service-role/producer-stepfunction-role
  1. After successful creation of the resources, go to the consumer account S3 bucket, consumer-databucket-<consumer account id>, and update the bucket policy as follows:
{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Sid": "AllowAppFlowDestinationActions",
            "Effect": "Allow",
            "Principal": {"Service": "appflow.amazonaws.com"},
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::consumer-databucket-<consumer-account-id>",
                "arn:aws:s3:::consumer-databucket-<consumer-account-id>/*"
            ]
        }, {
            "Sid": "Producer-stepfunction-role",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<producer-account-id>:role/service-role/producer-stepfunction-role"
            },
            "Action": [
                "s3:ListBucket",
                "s3:GetObject",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::consumer-databucket-<consumer-account-id>",
                "arn:aws:s3:::consumer-databucket-<consumer-account-id>/*"
            ]
        }
    ]
}

Validate the workflow

Let’s walk through the flow:

  1. Review the Marketo and Salesforce connection setup in the producer and consumer account respectively.

In the architecture section, we suggested scheduling the AppFlow (producer-flow) in the producer account. However, for quick testing purposes, we demonstrate how to manually run the flow on demand.

  1. Go to the AppFlow (producer-flow) in the producer account. On the Filters tab of the flow, choose Edit filters.
  2. Choose the Created At date range for which you have data.
  3. Save the range and choose Run flow.
  4. Review the producer S3 bucket.

AppFlow generates the files in the producer-flow prefix within this bucket. The files are temporarily located in the producer S3 bucket under s3://<producer-bucket>.<region>.<account-id>/producer-flow.

  1. Review the EventBridge rule and Step Functions state machine in the producer account.

The Amazon AppFlow job completion triggers an EventBridge rule (arn:aws:events:<region>:<producer account id>:rule/producer-appflow-completion-event, as noted in the Outputs tab of the CloudFromation stack in the Producer Account), which triggers the Step Functions state machine (arn:aws:states:<region>:<producer account id>:stateMachine:ProducerStateMachine-xxxx) in the producer account. The state machine copies the files to the consumer S3 bucket from the producer-flow prefix in the producer S3 bucket. Once file copy is complete, the state machine moves the files from the producer-flow prefix to the archive prefix in the producer S3 bucket. You can find the files in s3://<producer-bucket>.<region>.<account-id>/archive.

  1. Review the consumer S3 bucket.

The Step Functions state machine in the producer account copies the files to the consumer S3 bucket and sends an event to EventBridge in the consumer account. The files are located in the consumer S3 bucket under s3://consumer-databucket-<account-id>/marketo-leads-source/.

  1. Review the EventBridge rule (arn:aws:events:<region>:<consumer account id>:rule/consumer-custom-event-bus/consumer-custom-event-bus-rule) in the consumer account, which should have triggered the Step Function workflow (arn:aws:states:<region>:<consumer account id>:stateMachine:consumer-state-machine).

The AWS Glue crawler (consumer-glue-crawler) runs to update the metadata followed by the AWS Glue job (consumer-glue-job), which curates the data by applying the Do not call filter. The curated files are placed in s3://consumer-databucket-<account-id>/marketo-leads-curated/. After data curation, the flow is started as part of the state machine.

  1. Review the Amazon AppFlow job (arn:aws:appflow:<region>:<consumer account id>:flow/consumer-appflow) run status in the consumer account.

Upon a successful run of the Amazon AppFlow job, the curated data files are moved to the s3://consumer-databucket-<account-id>/marketo-leads-processed/ folder and Salesforce is updated with the leads. Additionally, all the original source files are moved from s3://consumer-databucket-<account-id>/marketo-leads-source/ to s3://consumer-databucket-<account-id>/marketo-leads-archive/.

  1. Review the updated data in Salesforce.

You will see newly created or updated leads created by Amazon AppFlow.

Clean up

To clean up the resources created as part of this post, delete the following resources:

  1. Delete the resources in the producer account:
    • Delete the producer S3 bucket content.
    • Delete the CloudFormation stack.
  2. Delete the resources in the consumer account:
    • Delete the consumer S3 bucket content.
    • Delete the CloudFormation stack.

Summary

In this post, we showed how you can support a cross-account model to exchange data between different partners with different SaaS integrations using Amazon AppFlow. You can expand this idea to support multiple target accounts.

For more information, refer to Simplifying cross-account access with Amazon EventBridge resource policies. To learn more about Amazon AppFlow, visit Amazon AppFlow.

About the authors

Ramakant Joshi is an AWS Solutions Architect, specializing in the analytics and serverless domain. He has a background in software development and hybrid architectures, and is passionate about helping customers modernize their cloud architecture.

Debaprasun Chakraborty is an AWS Solutions Architect, specializing in the analytics domain. He has around 20 years of software development and architecture experience. He is passionate about helping customers in cloud adoption, migration and strategy.

Suraj Subramani Vineet is a Senior Cloud Architect at Amazon Web Services (AWS) Professional Services in Sydney, Australia. He specializes in designing and building scalable and cost-effective data platforms and AI/ML solutions in the cloud. Outside of work, he enjoys playing soccer on weekends.

AWS recognized as a Challenger in the 2023 Gartner Magic Quadrant for Analytics and Business Intelligence Platforms

Post Syndicated from Jose Kunnackal original https://aws.amazon.com/blogs/big-data/aws-recognized-as-a-challenger-in-the-2023-gartner-magic-quadrant-for-analytics-and-business-intelligence-platforms/

AWS has been named a Challenger in the 2023 Gartner Magic Quadrant for Analytics and Business Intelligence (ABI) Platforms. Previously, AWS was positioned as a Niche player in the Magic Quadrant for ABI platforms. The Gartner Magic Quadrant evaluates 20 ABI companies based on their Ability to Execute and Completeness of Vision.

In our view, this recognition in the Magic Quadrant reinforces the progress we have made by tirelessly innovating on behalf of our customers. And this is just the beginning.

Benefits of QuickSight

AWS built QuickSight from the ground up as a cloud BI service to overcome challenges customers faced with alternative offerings. QuickSight powers data-driven organizations with unified business intelligence at hyperscale. With QuickSight, organizations of any size can meet the analytical needs of all users from the same source of truth through modern interactive dashboards, paginated reports, embedded analytics, and natural language queries. Since introducing QuickSight in 2016, we have been on a journey to democratize access to data for everyone in an organization. In 2022 alone, QuickSight added more than 80 capabilities, making it easier for you to deliver valuable business insights throughout your organization, when and where needed.

Today, over 100,000 customers use QuickSight as their BI service. Organizations of all sizes are choosing QuickSight for their BI needs and enabling users to understand, visualize, and derive insights and predictions from data, regardless of technical expertise.

Review the Gartner Magic Quadrant

2023 Gartner Magic Quadrant for Analytics and Business Intelligence Platforms

Access a complimentary copy of the full report to see why Gartner positioned AWS as a Challenger, and dive deep into the strengths and cautions of AWS.

We are excited about our momentum, strong vision, and the pace at which we are enabling our customers to democratize access to data for everyone in their organization.

Gartner Disclaimer

Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

Gartner, Magic Quadrant for Analytics and Business Intelligence Platforms, Kurt Schlegel, Julian Sun, David Pidsley, Anirudh Ganeshan, Fay Fei, Aura Popa, Radu Miclaus, Edgar Macari, Kevin Quinn, Christopher Long, 5 April 2023

Gartner is a registered trademark and service mark and Magic Quadrant is a registered trademark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and are used herein with permission. All rights reserved

This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from Amazon Web Services, Inc.

About the Author

Jose Kunnackal is Director of Product Management for Amazon QuickSight, AWS’ cloud-native, fully managed BI service. Jose started his career with Motorola, writing software for telecom and first responder systems. Later he was Director of Engineering at Trilibis Mobile, where he built a SaaS mobile web platform using AWS services. Jose is excited by the potential of cloud technologies to help customers make the most of their data.

Starting a Career in Tech? Learn How Rapid7’s Emerging Talent Programmes Foster Long-Term Success

Post Syndicated from Rapid7 original https://blog.rapid7.com/2023/04/25/starting-a-career-in-tech-learn-how-rapid7s-emerging-talent-programmes-foster-long-term-success/

Starting a Career in Tech? Learn How Rapid7’s Emerging Talent Programmes Foster Long-Term Success

Rapid7’s Emerging Talent Programmes pave the way for early career professionals to have a successful career in tech. In Belfast, we offer both an Apprentice Programme and a Placement Programme to support new talent coming into the tech field.

The Apprentice Programme is designed for individuals with established careers that want to pivot into the cybersecurity field. In our Placement Programme, students spend a full year working as a member of one of our teams while enrolled in university, gaining valuable experience working on real projects.

Cybersecurity is constantly evolving, and we work diligently to stay ahead of attackers and create a more secure digital future for all. Emerging Talent Programme participants bring fresh and valuable perspectives to our business and are equipped with the resources and opportunities to develop valuable industry knowledge and experience.

We spoke with several team members who went through these programmes and went on to pursue full time roles at Rapid7, and invited them to share their experiences.

What was the biggest hurdle you have had to overcome when starting your career?

When I joined Rapid7, I had no prior IT experience. My background was in mechanics. I was initially very nervous that my lack of IT experience would put me at a disadvantage. However, I was fortunate to be offered an opportunity with Rapid7 and Belfast Met. Rapid7 provided me with exceptional training and a support programme with ongoing mentoring. This ensured that I never felt left behind.

In the last 2 years working with Rapid7, my knowledge and skills have grown far beyond what I had ever hoped for. This is a testament to the talented individuals I work with and their willingness to share their expertise and help others.

Mark Gottschalk, Technical Support Engineer II

Coming from university and other experiences where I would have completed projects in smaller groups, understanding how information is stored and communicated across a large company and stakeholders took some time. I look at it as a valuable learning opportunity as you gain insight into how decisions are made in software companies and how stakeholders in all areas of Rapid7’s business interact with each other.

Christopher Jennings, Assistant Product Manager, Platform

How did you know you wanted to work at Rapid7?

I was interested in the culture at Rapid7 when looking for placement opportunities. The effort by the company to look after employee welfare was very attractive, and quite unique compared to other companies.

Kate Wilson, User Experience Designer I

My software engineering placement at Rapid7 was a pivotal experience in my career journey, as it helped me to determine my desired career path and confirm my interest in the company. Rapid7 impressed me with its position as a leader in the cybersecurity industry, as well as its ownership of the highly regarded tool, Metasploit.

The company’s focus on culture and values during the hiring process, and the positive experience I had with the people I met made a strong impression on me. These factors, combined with my passion for software engineering, led me to conclude that Rapid7 would be an ideal place for me to continue my professional development and pursue my career goals.

Jonathan Hume, Software Engineer I

How has this programme helped you determine your career path?

This placement programme has been instrumental in helping me determine my career path. I now have a better understanding of what product managers do and how different stakeholders within a software business interact to progress Rapid7 forward.

Christopher Jennings, Assistant Product Manager, Platform

The apprenticeship programme at Rapid7 has been a transformative experience for me. I have a strong interest in technology and a drive to learn and grow. I am surrounded by colleagues who are supportive and inclusive, and I have the opportunity to work on real-world projects that are challenging and rewarding. The programme provides a perfect balance between work and education, allowing me to complete my degree while gaining valuable experience.

Toni Ashe, Technical Support Engineer I

What has been your proudest accomplishment at Rapid7 so far?

My proudest accomplishment within Rapid7 has been the success of the Apprentice team I started with. At the end of our first year working within the Support team, we received the top three highest average customer survey ratings globally, as well as the top team globally. This was a moment of immense pride for all of us and serves as a testament to the effectiveness of the training and development programme for Support Engineers and the Apprenticeship programme!

Mark Gottschalk, Technical Support Engineer II, Support Coordinator Apprentice

My proudest accomplishment so far has been successfully completing my first coding project, which was a command palette modal that utilized hotkeys and other actions. Despite the challenges, I have worked hard and pushed myself to learn, and the reward of completing the project was worth all the effort.

Toni Ashe, Technical Support Engineer I

What piece of advice would you give someone entering a similar position?

I would advise anyone who is thinking about applying for an internship to be confident in their ability, ask lots of questions, and build a good knowledge base on the work and environment of the company and role you are applying to.

Kate Wilson, User Experience Designer I

For anyone considering a career move, my advice is: Take advantage of the opportunities for learning and growth, be open to feedback, and don’t be afraid to ask questions. Pursuing a career in this field can be challenging, but the results are rewarding if you are passionate about technology and have a drive to constantly learn and grow.

Toni Ashe, Technical Support Engineer I

Looking for a place where you can gain hands-on experience and drive real impact? Explore all our Emerging Talent programmes and opportunities here.

The collective thoughts of the interwebz