Седмицата (3–8 април)

Post Syndicated from Светла Енчева original https://www.toest.bg/sedmitsata-3-8-april/

Седмицата (3–8 април)

В „Тоест“ харесваме статии, които остаряват красиво. Ще ни се написаното в тях да бъде валидно и след години. Същевременно се стараем и да бъдем актуални. Трудно е да коментираме и анализираме последните парламентарни избори от 2 април обаче, без думите ни да остареят бързо, безславно и неестетично. Защото след петите избори за последните две години отново нищо не е ясно – нито дали ще има редовно правителство, а още по-малко, ако такова се сформира, какво ще бъде то и кой ще участва в него.

Първата статия в „Тоест“ след изборите по график се падна на мен, а спонтанната ми реакция на резултатите от тях беше „Пак ли, бе?“. Не мисля, че тази публикация ще остарее изящно, макар в нея да се опитах да откроя и някои трайни проблеми, например неспособността на демократичните и реформистките партии да отправят послания към избирателите, които са извън твърдия им електорат. А за да имаш послание, трябва да познаваш и разбираш тези хора.

За да бъде максимално актуална, Емилия Милчева изчака пресконференцията на коалицията ПП–ДБ на 7 април. „Заровете за правителство още се търкалят“, казва тя, но предполага, че по-голяма е вероятността да се стигне до шести поредни парламентарни избори. Защото Бойко Борисов иска първо да се прави коалиция, а после да се говори за политики, пък от ПП–ДБ настояват на обратното – политиките са важни, ако за тях има консенсус, правителство може да има и без коалиция.

Ако след вота нещо се очертава като устойчива тенденция, това е ръстът в подкрепата за „Възраждане“. Една от предпоставките за него е безкритичното отношение на мейнстрийм медиите както към крайнодясната идеология на партията на Костадин Костадинов, към системно тиражираните фалшиви новини и език на омразата, така и към агресията спрямо журналисти. Само в рамките на предизборната кампания от „Възраждане“ проявиха подобна агресия поне два пъти.

Активисти на партията не позволиха на Емилия Милчева да снима за свой материал в „Дойче Веле“ публично събитие – събиране на подписи за референдума против еврото. В последния ден от кампанията пък журналистката от БНТ Цвета (Флора) Стратиева беше изгонена с обиди и заплахи от пресконференция на партията. БНТ коментира случая едва след изборите.

„Възраждане“ увеличава популярността си и сред българите в чужбина. В статията си „Електоралната енигма – защо българите в чужбина гласуват за „Възраждане“ Марина Лякова отговаря на какво се дължи тази популярност. Тя описва типажите, които харесват партията на Костадинов, техните проблеми и мотиви, трудностите им да се впишат в приемащите общества. Според нея в подкрепата за „Възраждане“ всъщност няма нищо енигматично. Ала българите в чужбина, които гласуват за националистическа и антиевропейска партия, все едно режат клона, на който седят, вярвайки, че ще полетят в облаците.

Както в политиката, така и в частния живот неспособността да се води спокоен разговор е причина за много проблеми. Но ако в българската политика нямаме независим медиатор, на когото да се доверим, при определени частни спорове и конфликти имаме тази възможност. Във втората си статия по темата, озаглавена „Медиацията на практика“, съдия Владимир Вълков ни разказва как всъщност се постига извънсъдебно споразумение посредством медиация. Тази процедура е доброволна, а целта е да се стигне до решение, приемливо и за двете страни.

Продължаваме с друга статия на съдия – на Калин Калпакчиев, която може да се разглежда като логично свързана с темата за медиацията – „Как възстановителният подход може да се съчетае с традиционното правосъдие“. Защото възстановителното правосъдие не е възможно, ако двете страни не са постигнали съгласие помежду си и не са намерили приемливо за тях решение. Съдия Калпакчиев ни разказва истинска история за двама изпаднали младежи, които пребили и ограбили възрастен мъж, но благодарение на възстановителното правосъдие получили възможност да поправят стореното, да допринесат за живота в селото си и изобщо – получили шанс за нов живот.

При религиозния фундаментализъм обаче спокойният разговор е невъзможен, особено когато религията се използва за политически цели. В продължението на статията си „Изображенията на Пророка Мохамед: Случаят в „Хамлин“ Атанас Шиников разсъждава какво се случва, когато изображение на Пророка Мохамед (когото, знаем, ислямът забранява да бъде изобразяван) се появи не като умишлена провокация, какъвто беше случаят с „Шарли Ебдо“, а присъства като артефакт в самата ислямска традиция. Всъщност ислямът е много по-шарен, отколкото изглежда както на мюсюлманските фундаменталисти, така и на ислямофобите.

За разлика от добрите статии, които остаряват красиво, добрата литература изобщо не остарява. В България ковчежето на нидерландската литература е неоткрехнато, твърди преводачката Мария Енчева. Тя ни показва три съкровища от това ковчеже, които се надява някога да преведе. Първото е романът „Никога вече сън“ от Вилем Фредерик Херманс, който е едно от най-превежданите произведения в нидерландската литература, а само в родната си страна има 39 издания. Вторият роман е озаглавен „Всичко отдавна отминало“ и е на младата хомосексуална авторка Хана Берфутс. Третото съкровище не е книга, а автор – Ремко Кемперт. Той пише поезия, романи и новели. Няколко месеца преди смъртта му през 2022 г. „Тоест“ помести на стената си във Facebook откъс от стихотворението му „Съпротива“.

Знаете си, че няма да ви оставя без лична препоръка, нали? Започнах с изборите, но ще завърша със… секс.

Като говорим за красиво остаряване – британската актриса Ема Томпсън определено остарява красиво. Може да се уверим в това, ако видим ролята ѝ във филма „Успех, Лио Гранд“ (Good Luck to You, Leo Grande, 2022 г.). Ако го гледате, при последната сцена ще си спомните за красивото остаряване. Томпсън влиза в ролята на пенсионирана учителка по религия, която след смъртта на мъжа си решава да се опита да разбере какво толкова намират хората в секса. За тази цел наема млад секс работник (очарователно изигран от ирландския актьор Дарил Маккормак). За да постигне сексуално удоволствие обаче, тя трябва да се пребори със собствените си бариери, да се научи да приема и желанията си, и тялото си.

Филмът е с брилянтни диалози, смешно-сериозен и… много секси. Той повдига една пренебрегвана тема – когато една жена мине 60-годишна възраст, това не означава, че сексуалните ѝ потребности изчезват. Ала докато комбинацията между възрастен мъж и млада жена е класика, тази между възрастна жена и млад мъж се смята за неприемлива (спомнете си какво се изговори за Брижит Макрон, съпругата на френския президент Еманюел Макрон).

А танцът на Томпсън и Маккормак се нарежда сред любимите ми филмови танци, наред с този на Кристина Ричи под звуците на King Crimson в „Бъфало 66“ (но в „Бъфало 66“, трябва да призная, ролята на жената е… проблематична):

https://www.youtube.com/watch?v=Piws-ROqJF0

Да, „Успех, Лио Гранд“ среща справедливи критики, че не представя толкова позицията на секс работника, колкото на клиентката му. Човек, за когото най-голямото щастие е да доставя удоволствие на другите, е идеалният секс работник, нали? Реалните обаче имат и други приоритети (и проблеми), освен да задоволяват клиентите си. Но – без да пренебрегваме секс работниците – темата във филма е потребността на жените над 60 от сексуално удоволствие. Нека поне веднъж сексуалността на възрастните жени да е на първо място, а?

Тази седмица се събра доста за четене, но пък следващата „Тоест“ излиза в кратка ваканция. Очаквайте нови статии след Великден. Приятно четене и (ако съм събудила интереса ви) гледане! И весели празници!

Introducing AWS Lambda response streaming

Post Syndicated from Julian Wood original https://aws.amazon.com/blogs/compute/introducing-aws-lambda-response-streaming/

Today, AWS Lambda is announcing support for response payload streaming. Response streaming is a new invocation pattern that lets functions progressively stream response payloads back to clients.

You can use Lambda response payload streaming to send response data to callers as it becomes available. This can improve performance for web and mobile applications. Response streaming also allows you to build functions that return larger payloads and perform long-running operations while reporting incremental progress.

In traditional request-response models, the response needs to be fully generated and buffered before it is returned to the client. This can delay the time to first byte (TTFB) performance while the client waits for the response to be generated. Web applications are especially sensitive to TTFB and page load performance. Response streaming lets you send partial responses back to the client as they become ready, improving TTFB latency to within milliseconds. For web applications, this can improve visitor experience and search engine rankings.

Other applications may have large payloads, like images, videos, large documents, or database results. Response streaming lets you transfer these payloads back to the client without having to buffer the entire payload in memory. You can use response streaming to send responses larger than Lambda’s 6 MB response payload limit up to a soft limit of 20 MB.

Response streaming currently supports the Node.js 14.x and subsequent managed runtimes. You can also implement response streaming using custom runtimes. You can progressively stream response payloads through Lambda function URLs, including as an Amazon CloudFront origin, along with using the AWS SDK or using Lambda’s invoke API. You can also use Amazon API Gateway and Application Load Balancer to stream larger payloads.

Writing response streaming enabled functions

Writing the handler for response streaming functions differs from typical Node handler patterns. To indicate to the runtime that Lambda should stream your function’s responses, you must wrap your function handler with the streamifyResponse() decorator. This tells the runtime to use the correct stream logic path, allowing the function to stream responses.

This is an example handler with response streaming enabled:

exports.handler = awslambda.streamifyResponse(
    async (event, responseStream, context) => {
        responseStream.setContentType(“text/plain”);
        responseStream.write(“Hello, world!”);
        responseStream.end();
    }
);

The streamifyResponse decorator accepts the following additional parameter, responseStream, besides the default node handler parameters, event, and context.

The new responseStream object provides a stream object that your function can write data to. Data written to this stream is sent immediately to the client. You can optionally set the Content-Type header of the response to pass additional metadata to your client about the contents of the stream.

Writing to the response stream

The responseStream object implements Node’s Writable Stream API. This offers a write() method to write information to the stream. However, we recommend that you use pipeline() wherever possible to write to the stream. This can improve performance, ensuring that a faster readable stream does not overwhelm the writable stream.

An example function using pipeline() showing how you can stream compressed data:

const pipeline = require("util").promisify(require("stream").pipeline);
const zlib = require('zlib');
const { Readable } = require('stream');

exports.gzip = awslambda.streamifyResponse(async (event, responseStream, _context) => {
    // As an example, convert event to a readable stream.
    const requestStream = Readable.from(Buffer.from(JSON.stringify(event)));
    
    await pipeline(requestStream, zlib.createGzip(), responseStream);
});

Ending the response stream

When using the write() method, you must end the stream before the handler returns. Use responseStream.end() to signal that you are not writing any more data to the stream. This is not required if you write to the stream with pipeline().

Reading streamed responses

Response streaming introduces a new InvokeWithResponseStream API. You can read a streamed response from your function via a Lambda function URL or use the AWS SDK to call the new API directly.

Neither API Gateway nor Lambda’s target integration with Application Load Balancer support chunked transfer encoding. It therefore does not support faster TTFB for streamed responses. You can, however, use response streaming with API Gateway to return larger payload responses, up to API Gateway’s 10 MB limit. To implement this, you must configure an HTTP_PROXY integration between your API Gateway and a Lambda function URL, instead of using the LAMBDA_PROXY integration.

You can also configure CloudFront with a function URL as origin. When streaming responses through a function URL and CloudFront, you can have faster TTFB performance and return larger payload sizes.

Using Lambda response streaming with function URLs

You can configure a function URL to invoke your function and stream the raw bytes back to your HTTP client via chunked transfer encoding. You configure the Function URL to use the new InvokeWithResponseStream API by changing the invoke mode of your function URL from the default BUFFERED to RESPONSE_STREAM.

RESPONSE_STREAM enables your function to stream payload results as they become available if you wrap the function with the streamifyResponse() decorator. Lambda invokes your function using the InvokeWithResponseStream API. If InvokeWithResponseStream invokes a function that is not wrapped with streamifyResponse(), Lambda does not stream the response and instead returns a buffered response which is subject to the 6 MB size limit.

Using AWS Serverless Application Model (AWS SAM) or AWS CloudFormation, set the InvokeMode property:

  MyFunctionUrl:
    Type: AWS::Lambda::Url
    Properties:
      TargetFunctionArn: !Ref StreamingFunction
      AuthType: AWS_IAM
      InvokeMode: RESPONSE_STREAM

Using generic HTTP client libraries with function URLs

Each language or framework may use different methods to form an HTTP request and parse a streamed response. Some HTTP client libraries only return the response body after the server closes the connection. These clients do not work with functions that return a response stream. To get the benefit of response streams, use an HTTP client that returns response data incrementally. Many HTTP client libraries already support streamed responses, including the Apache HttpClient for Java, Node’s built-in http client, and Python’s requests and urllib3 packages. Consult the documentation for the HTTP library that you are using.

Example applications

There are a number of example Lambda streaming applications in the Serverless Patterns Collection. They use AWS SAM to build and deploy the resources in your AWS account.

Clone the repository and explore the examples. The README file in each pattern folder contains additional information.

git clone https://github.com/aws-samples/serverless-patterns/ 
cd serverless-patterns

Time to first byte using write()

  1. To show how streaming improves time to first bite, deploy the lambda-streaming-ttfb-write-sam pattern.
    2. cd lambda-streaming-ttfb-write-sam
  2. Use AWS SAM to deploy the resources to your AWS account. Run a guided deployment to set the default parameters for the first deployment.
    3. sam deploy -g --stack-name lambda-streaming-ttfb-write-sam

    For subsequent deployments you can use sam deploy.

  3. Enter a Stack Name and accept the initial defaults.

    4. AWS SAM deploys a Lambda function with streaming support and a function URL.

    AWS SAM deploy --g

    AWS SAM deploy –g

    Once the deployment completes, AWS SAM provides details of the resources.

    AWS SAM resources

    AWS SAM resources

    The AWS SAM output returns a Lambda function URL.

  4. Use curl with your AWS credentials to view the streaming response as the URL uses AWS Identity and Access Management (IAM) for authorization. Replace the URL and Region parameters for your deployment.
curl --request GET https://<url>.lambda-url.<Region>.on.aws/ --user AKIAIOSFODNN7EXAMPLE:wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY --aws-sigv4 'aws:amz:<Region>:lambda'

You can see the gradual display of the streamed response.

Using curl to stream response from write () function

Using curl to stream response from write () function

Time to first byte using pipeline()

  1. To try an example using pipeline(), deploy the lambda-streaming-ttfb-pipeline-sam pattern.
    2. cd ..
cd lambda-streaming-ttfb-pipeline-sam
  2. Use AWS SAM to deploy the resources to your AWS account. Run a guided deployment to set the default parameters for the first deploy.
    3. sam deploy -g --stack-name lambda-streaming-ttfb-pipeline-sam
  3. Enter a Stack Name and accept the initial defaults.
  4. Use curl with your AWS credentials to view the streaming response. Replace the URL and Region parameters for your deployment.
curl --request GET https://<url>.lambda-url.<Region>.on.aws/ --user AKIAIOSFODNN7EXAMPLE:wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY --aws-sigv4 'aws:amz:<Region>:lambda'

You can see the pipelined response stream returned.

Using curl to stream response from function

Using curl to stream response from function

Large payloads

  1. To show how streaming enables you to return larger payloads, deploy the lambda-streaming-large-sam application. AWS SAM deploys a Lambda function, which returns a 7 MB PDF file which is larger than Lambda’s non-stream 6 MB response payload limit.
    2. cd ..
cd lambda-streaming-large-sam
sam deploy -g --stack-name lambda-streaming-large-sam
  2. The AWS SAM output returns a Lambda function URL. Use curl with your AWS credentials to view the streaming response.
curl --request GET https://<url>.lambda-url.<Region>.on.aws/ --user AKIAIOSFODNN7EXAMPLE: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY --aws-sigv4 'aws:amz:<Region>:lambda' -o SVS401-ri22.pdf -w '%{content_type}'

This downloads the PDF file SVS401-ri22.pdf to your current directory and displays the content type as application/pdf.

You can also use API Gateway to stream a large payload with an HTTP_PROXY integration with a Lambda function URL.

Invoking a function with response streaming using the AWS SDK

You can use the AWS SDK to stream responses directly from the new Lambda InvokeWithResponseStream API. This provides additional functionality such as handling midstream errors. This can be helpful when building, for example, internal microservices. Response streaming is supported with the AWS SDK for Java 2.x, AWS SDK for JavaScript v3, and AWS SDKs for Go version 1 and version 2.

The SDK response returns an event stream that you can read from. The event stream contains two event types. PayloadChunk contains a raw binary buffer with partial response data received by the client. InvokeComplete signals that the function has completed sending data. It also contains additional metadata, such as whether the function encountered an error in the middle of the stream. Errors can include unhandled exceptions thrown by your function code and function timeouts.

Using the AWS SDK for Javascript v3

  1. To see how to use the AWS SDK to stream responses from a function, deploy the lambda-streaming-sdk-sam pattern.
    2. cd ..
cd lambda-streaming-sdk-sam
sam deploy -g --stack-name lambda-streaming-sdk-sam
  2. Enter a Stack Name and accept the initial defaults.

    3. AWS SAM deploys three Lambda functions with streaming support.

  • HappyPathFunction: Returns a full stream.
  • MidstreamErrorFunction: Simulates an error midstream.
  • TimeoutFunction: Function times out before stream completes.
  • Run the SDK example application, which invokes each Lambda function and outputs the result.
    • npm install @aws-sdk/client-lambda
node index.mjs

    You can see each function and how the midstream and timeout errors are returned back to the SDK client.

    Streaming midstream error

    Streaming midstream error

    Streaming timeout error

    Streaming timeout error

    Quotas and pricing

    Streaming responses incur an additional cost for network transfer of the response payload. You are billed based on the number of bytes generated and streamed out of your Lambda function over the first 6 MB. For more information, see Lambda pricing.

    There is an initial maximum response size of 20 MB, which is a soft limit you can increase. There is a maximum bandwidth throughput limit of 16 Mbps (2 MB/s) for streaming functions.

    Conclusion

    Today, AWS Lambda is announcing support for response payload streaming to send partial responses to callers as the responses become available. This can improve performance for web and mobile applications. You can also use response streaming to build functions that return larger payloads and perform long-running operations while reporting incremental progress. Stream partial responses through Lambda function URLs, or using the AWS SDK. Response streaming currently supports the Node.js 14.x and subsequent runtimes, as well as custom runtimes.

    There are a number of example Lambda streaming applications in the Serverless Patterns Collection to explore the functionality.

    Lambda response streaming support is also available through many AWS Lambda Partners such as Datadog, Dynatrace, New Relic, Pulumi and Lumigo.

    For more serverless learning resources, visit Serverless Land.

    Friday Squid Blogging: Squid Food Poisoning

    Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/friday-squid-blogging-squid-food-poisoning.html

    University of Connecticut basketball player Jordan Hawkins claims to have suffered food poisoning from calamari the night before his NCAA finals game. The restaurant disagrees:

    On Sunday, a Mastro’s employee politely cast doubt on the idea that the restaurant might have caused the illness, citing its intense safety protocols. The staffer, who spoke on condition of anonymity because he was not authorized to officially speak for Mastro’s, said restaurants in general were more likely to arouse suspicion when they had some rooting interest against the customer-athletes.

    As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.

    Read my blog posting guidelines here.

    Metasploit Weekly Wrap-Up

    Post Syndicated from Jeffrey Martin original https://blog.rapid7.com/2023/04/07/metasploit-weekly-wrap-up-5/

    The tide rolls in and out.

    Metasploit Weekly Wrap-Up

    The flood of new modules last week crested leaving ample time for documentation updates this week. The team and the community seem to have focused on getting those sweet sprinkles of information that help everyone understand Metasploit out to the world.

    Enhancements and features (1)

    • #17458 from steve-embling – Updates the exploit/multi/misc/weblogic_deserialize_badattrval module to enable support for SSL/TLS.

    Bugs fixed (4)

    • #17778 from adfoster-r7 – Updates the Metasploit database migration code to no longer break the test suite when running locally.
    • #17823 from bcoles – This fixes an issue in the check method where targets with files containing no PHP code were falsely reported as safe.
    • #17835 from bcoles – Fixes a bug in auxiliary/admin/networking/cisco_dcnm_auth_bypass where the bypass_auth method would break if a user supplied a TARGETURI path without a trailing /.
    • #17844 from SubcomandanteMeowcos – Fixes broken documentation references in the secretsdump, zemra_panel_rce, and windows/gather/credentials/skype modules.

    Documentation added (6)

    • #17836 from jheysel-r7 – Documents the usage of session.platform in the How to get started with writing a post-module documentation.
    • #17837 from cdelafuente-r7 – Updates the ‘How to write a check method’ page to include using the Msf::Exploit::Remote::AutoCheck mixin, which will automatically run a check method against a target before attempting to exploit it.
    • #17838 from zeroSteiner – Updates the How to use railgun for windows post exploitation documentation with the latest conventions for using Meterpreter’s Railgun when wanting to interact with Windows APIs on a remote target.
    • #17840 from jheysel-r7 – Updates the ‘Get started writing an Exploit’ example documentation to describe the usage of Stability/Reliability/SideEffects metadata when writing modules.
    • #17841 from jheysel-r7 – Documents the latest labels that can be assigned to pull requests.
    • #17842 from bwatters-r7 – Updates the How to use command stagers documentation with additional examples and clearer descriptions.

    You can always find more documentation on our docsite at docs.metasploit.com.

    Get it

    As always, you can update to the latest Metasploit Framework with msfupdate
    and you can get more details on the changes since the last blog post from
    GitHub:

    If you are a git user, you can clone the Metasploit Framework repo (master branch) for the latest.
    To install fresh without using git, you can use the open-source-only Nightly Installers or the
    binary installers (which also include the commercial edition).

    Why 2 DIMMs Per Channel Will Matter Less in Servers

    Post Syndicated from Patrick Kennedy original https://www.servethehome.com/why-2-dimms-per-channel-will-matter-less-in-servers/

    We go into why 2 DIMMs per channel (2DPC) designs are going to be less common in future server designs due to a number of current constraints

    The post Why 2 DIMMs Per Channel Will Matter Less in Servers appeared first on ServeTheHome.

    Заровете за правителство още се търкалят

    Post Syndicated from Емилия Милчева original https://www.toest.bg/zarovete-za-pravitelstvo-oshte-se-turkaliyat/

    Заровете за правителство още се търкалят

    Има ли правителство на хоризонта и победителят на изборите ГЕРБ ли ще зададе формулата? Отказът на втората политическа сила „Продължаваме промяната“ – „Демократична България“ да управлява в съюз с ГЕРБ увеличи шансовете за нови избори през лятото.

    Лидерът Бойко Борисов настоя първо да се постигне съгласие за управление, а после да се обсъждат приоритети. Обратно на логичния ход първо да се изгради широка платформа за съществените задачи за разрешаване и на тази база да се търси подкрепа. В нова роля – на лидер с разум и благост в гласа, и с нова профилна снимка, на която е усмихнат многозначително, Борисов като Вазовия чорбаджи Марко сбира политиците, за да направят най-доброто за България – редовно правителство. Даже покани и президента, защото управлявал особено дълго, и в известна степен унижи парламентаризма, побългарявайки го.

    В 49-тия парламент Бойко Борисов ще е депутат и този път лично ще води преговорите, но премиер едва ли ще е.

    Обръщам се към лидерите на партиите: по време на кампания се чертаят разделителни линии, защото иначе всички ще са еднакви, но след изборите в демократичния свят, ако на някоя партия не ѝ стигат 121 гласа, се правят коалиции. Без редовно правителство бюджетът няма да бъде подкрепен от нас,

    каза Борисов на първата си пресконференция след изборите.

    В капана на Бойко Борисов

    Така водачът на ГЕРБ не остави много полезни ходове на лидерите на втората политическа сила ПП–ДБ, изостанала с около 2% от ГЕРБ–СДС. Влязат ли в съюз с ГЕРБ, ще загубят още от избирателите си на местните избори, не влязат ли – губят шансове за мобилизация на избиратели от периферията, извън ядрата си (доколкото ПП например има такива). ПП–ДБ твърде дълго мълчаха след изборите, проговаряйки едва на петия ден – ако не броим порива на „Продължаваме промяната“ да споделят предварително позицията си, че няма да се коалират с ГЕРБ. Общото решение на коалицията, съобщено в петък, го подпечата.

    „Оценяваме и си даваме сметка, че голяма част от българските граждани, около 80%, искат да се прекрати политическата криза, да бъде създадено редовно правителството, защото пред държавата има много предизвикателства. Ние сме солидарни с тях, но решението на нашето национално ръководство е, че не можем да подкрепим правителство, кадрувано от Бойко Борисов“, заяви съпредседателят на ДБ и лидер на ДСБ Атанас Атанасов.

    Другият съпредседател на ДБ – Христо Иванов, потвърди общата позиция на коалицията и заяви, че „оттук нататък следва изпълнението на най-важното обещание – да бъде възстановена парламентарната република и парламентарното управление в България, и то по начин, който може да реализира приоритетите ни на практика, не на думи“.

    По политики обаче ще работят с всички политически партии, както обясни съпредседателят на ПП Кирил Петков – а това значи и с ГЕРБ. „Ако с първия мандат ГЕРБ не успее да формира правителство, ние ще поемем отговорността и ще предложим с втория мандат правителство от нашата коалиция“, каза Петков. Водената от Бойко Борисов партия едва ли би го подкрепила – освен ако не иска да му издърпа килима под краката в определен момент.

    За политики може, за коалиция – не

    „Коалиция няма да има, което не значи, че няма да има правителство“, заяви по БНР тази седмица евродепутатът Радан Кънев (ДСБ/ЕНП). „Вариант за управляваща коалиция не виждам. Не означава обаче, че няма формула за съставяне на редовен кабинет. Има формули за съставяне на правителство, които не са политическа коалиция“, обясни Кънев. Има. Което оставя вратичката пред ПП–ДБ да обявят частична подкрепа за политики – еврозоната, Шенген, военна подкрепа за Украйна – и за приоритети: приемане на закона за държавния бюджет за 2023 г., на законите, които са необходими, за да получи България втория транш по Плана за възстановяване и устойчивост (с които непростимо изостана), на програмите за модернизация на армията, за службите и др.

    ПП–ДБ смята да постави и въпроса за съдебната реформа и съдбата на главния прокурор Иван Гешев. По-рано Борисов беше казал, че още в първите дни на новия парламент ще настоява за приемането на „закона на Крум Зарков (служебния правосъден министър – б.а.) за корупцията“ и за механизма за разследване на главния прокурор – „Ще приемем този закон, даже и да е противоконституционен, три пъти за пет дни ходих в Американското посолство“.

    Така хем няма да участват в правителство на ГЕРБ, но и няма да отблъснат напълно тези избиратели, които смятат, че е необходимо да се положат усилия за редовно правителство, за да бъде спряно царуването на президента Радев. Коалицията избра да се вслуша в непримиримите за какъвто и да е съюз с ГЕРБ заради „ценностни различия“ (по Кирил Петков), като не допусна никакви въпроси на журналисти след изявленията на лидерите в петък.

    Оттук нататък големият въпрос е дали Бойко Борисов може да направи правителство на малцинството с подкрепа на останалите политически сили и какво ще си гарантира в замяна. Той вече заяви публично, че кабинет с БСП и „Възраждане“ няма да прави, независимо че ги покани за разговори. ГЕРБ можеше да направи правителство с БСП и ДПС и в 48-мия парламент, но не пожела. В бъдещия 49-ти парламент ГЕРБ–СДС получава 69 мандата, 64 са за ПП–ДБ, 37 за „Възраждане“, 36 за ДПС. БСП ще има 23-ма депутати, а ИТН – 11. Тоест ГЕРБ–СДС заедно с ДПС и ИТН ще имат общо 116 гласа, които не са мнозинство от 121 народни представители, но може да осигурят добра подкрепа за евентуално правителство.

    Ако няма правителство

    Какво би станало, ако отново се отиде на избори? На първо място, закрепва се владичеството на президента в ситуация, усложнена в геостратегически план от войната в Украйна. Румен Радев, както е известно, не споделя позициите на ЕС и НАТО за военна помощ за Киев – от особено значение, когато украинската армия готви контраофанзива.

    Още избиратели ще се разбягат като прелетни птици, като удара ще понесат основно ПП, които бездруго нямат устойчиви партийни структури. На изборите на 2 април коалицията между „Продължаваме промяната“ и „Демократична България“ вече изгуби 73 000 гласа в сравнение с резултатите им от предходния вот, като спад се отбелязва и в България, и в чужбина.

    България няма да успее да получи втория транш по Плана за възстановяване и устойчивост – средства, предназначени за зелена и цифрова трансформация на икономиката, и отново ще е най-изоставащата.

    На следващите избори проруската „Възраждане“, в чиито послания се откриват елементи от нацистката пропаганда, има потенциал да стане втора политическа сила. В подписката за референдум за запазване на лева (на практика срещу еврото) са събрани почти 600 000 подписа и ще бъдат внесени в Народното събрание.

    Предстои политическите сили в парламента да направят своя избор и да поемат отговорността за него.

    [$] The shrinking role of semaphores

    Post Syndicated from original https://lwn.net/Articles/928026/

    The kernel’s handling of concurrency has changed a lot over the years. In
    2023, a kernel developer’s toolkit includes tools like completions, highly
    optimized mutexes, and a variety of lockless
    algorithms    . But, once upon a time, concurrency control came down to
    the use of simple semaphores; a discussion on a small change to the
    semaphore API shows just how much the role of semaphores has changed over
    the course of the kernel’s history.

    Push Amazon EMR step logs from Amazon EC2 instances to Amazon CloudWatch logs

    Post Syndicated from Nausheen Sayed original https://aws.amazon.com/blogs/big-data/push-amazon-emr-step-logs-from-amazon-ec2-instances-to-amazon-cloudwatch-logs/

    Amazon EMR is a big data service offered by AWS to run Apache Spark and other open-source applications on AWS to build scalable data pipelines in a cost-effective manner. Monitoring the logs generated from the jobs deployed on EMR clusters is essential to help detect critical issues in real time and identify root causes quickly.

    Pushing those logs into Amazon CloudWatch enables you to centralize and drive actionable intelligence from your logs to address operational issues without needing to provision servers or manage software. You can instantly begin writing queries with aggregations, filters, and regular expressions. In addition, you can visualize time series data, drill down into individual log events, and export query results to CloudWatch dashboards.

    To ingest logs that are persisted on the Amazon Elastic Compute Cloud (Amazon EC2) instances of an EMR cluster into CloudWatch, you can use the CloudWatch agent. This provides a simple way to push logs from an EC2 instance to CloudWatch.

    The CloudWatch agent is a software package that autonomously and continuously runs on your servers. You can install and configure the CloudWatch agent to collect system and application logs from EC2 instances, on-premises hosts, and containerized applications. CloudWatch processes and stores the logs collected by the CloudWatch agent, which further helps with the performance and health monitoring of your infrastructure and applications.

    In this post, we create an EMR cluster and centralize the EMR step logs of the jobs in CloudWatch. This will make it easier for you to manage your EMR cluster, troubleshoot issues, and monitor performance. This solution is particularly helpful if you want to use CloudWatch to collect and visualize real-time logs, metrics, and event data, streamlining your infrastructure and application maintenance.

    Overview of solution

    The solution presented in this post is based on a specific configuration where the EMR step concurrency level is set to 1. This means that only one step is run at a time on the cluster. It’s important to note that if the EMR step concurrency level is set to a value greater than 1, the solution may not work as expected. We highly recommend verifying your EMR step concurrency configuration before implementing the solution presented in this post.

    The following diagram illustrates the solution architecture.

    Solution Architecture Diagram

    The workflow includes the following steps:

    1. Users start an Apache Spark EMR job, creating a step on the EMR cluster. Using Apache Spark, the workload is distributed across the different nodes of the EMR cluster.
    2. In each node (EC2 instance) of the cluster, a CloudWatch agent watches different logs directories, capturing new entries in the log files and pushing them to CloudWatch.
    3. Users can view the step logs accessing the different log groups from the CloudWatch console. The step logs written by Amazon EMR are as follows:
      • controller — Information about the processing of the step. If your step fails while loading, you can find the stack trace in this log.
      • stderr — The standard error channel of Spark while it processes the step.
      • stdout — The standard output channel of Spark while it processes the step.

    We provide an AWS CloudFormation template in this post as a general guide. The template demonstrates how to configure a CloudWatch agent on Amazon EMR to push Spark logs to CloudWatch. You can review and customize it as needed to include your Amazon EMR security configurations. As a best practice, we recommend including your Amazon EMR security configurations in the template to encrypt data in transit.

    You should also be aware that some of the resources deployed by this stack incur costs when they remain in use.

    In the next sections, we go through the following steps:

    1. Create and upload the bootstrap script to an Amazon Simple Storage Service (Amazon S3) bucket.
    2. Use the CloudFormation template to create the following resources:
    3. Monitor the Spark logs on the CloudWatch console.

    Prerequisites

    This post assumes that you have the following:

    Create and upload the bootstrap script to an S3 bucket

    For more information, see Uploading objects and Installing and running the CloudWatch agent on your servers.

    To create and the upload the bootstrap script, complete the following steps:

    1. Create a local file named bootstrap_cloudwatch_agent.sh with the following content: 
      #!/bin/bash

echo -e 'Installing CloudWatch Agent... \n'
sudo rpm -Uvh --force https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm

echo -e 'Starting CloudWatch Agent... \n'
sudo amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c ssm:AmazonCloudWatch-Config.json -s

    2. On the Amazon S3 console, choose your S3 bucket.
    3. On the Objects tab, choose Upload.
    4. Choose Add files, then choose the bootstrap script.
    5. Choose Upload, then choose the file name: bootstrap_cloudwatch_agent.sh.
    6. Choose Copy S3 URI. We use this value in a later step.

    Provision resources with the CloudFormation template

    Choose Launch Stack to launch a CloudFormation stack in your account and deploy the template:

    This template creates an IAM role, IAM instance profile, Systems Manager parameter, and EMR cluster. The cluster starts the Spark PI estimation example application. You will be billed for the AWS resources used if you create a stack from this template.

    The CloudFormation wizard will ask you to modify or provide these parameters:

    • InstanceType – The type of instance for all instance groups. The default is m4.xlarge.
    • InstanceCountCore – The number of instances in the core instance group. The default is 2.
    • EMRReleaseLabel – The Amazon EMR release label you want to use. The default is emr-6.9.0.
    • BootstrapScriptPath – The S3 path of your CloudWatch agent installation bootstrap script that you copied earlier.
    • Subnet – The EC2 subnet where the cluster launches. You must provide this parameter.
    • EC2KeyPairName – An optional EC2 keypair for connecting to cluster nodes, as an alternative to Session Manager.

    Monitor the log streams

    After the CloudFormation stack deploys successfully, on the CloudWatch console, choose Log groups in the navigation pane. Then filter the log groups by the prefix /aws/emr/master.

    choose Log groups in the navigation pane

    The ID in the log group corresponds to the EC2 instance ID of the EMR primary node. If you have multiple EMR clusters, you can use this ID to identify a particular EMR cluster, based on the primary node ID.

    In the log group, you will find the three different log streams.

    In the log group, you will find the three different log streams.

    The log streams contain the following information:

    • step-stdout – The standard output channel of Spark while it processes the step.
      The standard output channel of Spark while it processes the step
    • step-stderr – The standard error channel of Spark while it processes the step.
      The standard error channel of Spark while it processes the step.
    • step-controller – Information about the processing of the step. If your step fails while loading, you can find the stack trace in this log.
      Information about the processing of the step.

    Clean up

    To avoid future charges in your account, delete the resources you created in this walkthrough. The EMR cluster will incur charges as long as the cluster is active, so stop it when you’re done.

    1. On the CloudFormation console, in the navigation pane, choose Stacks.
    2. Choose the stack you launched (EMR-CloudWatch-Demo), then choose Delete.
    3. Empty the S3 bucket you created.
    4. Delete the S3 bucket you created.

    Conclusion

    Now that you have completed the steps in this walkthrough, you have the CloudWatch agent running on your cluster hosts and configured to push EMR step logs to CloudWatch. With this feature, you can effectively monitor the health and performance of your Spark jobs running on Amazon EMR, detecting critical issues in real time and identifying root causes quickly.

    You can package and deploy this solution through a CloudFormation template like this example template, which creates the IAM instance profile role, Systems Manager parameter, and EMR cluster.

    To take this further, consider using these logs in CloudWatch alarms for alerts on a log group-metric filter. You could collect them with other alarms into a composite alarm or configure alarm actions such as sending Amazon Simple Notification Service (Amazon SNS) notifications to trigger event-driven processes such as AWS Lambda functions.

    About the Author

    Ennio Pastore is a Senior Data Architect on the AWS Data Lab team. He is an enthusiast of everything related to new technologies that have a positive impact on businesses and general livelihood. Ennio has over 10 years of experience in data analytics. He helps companies define and implement data platforms across industries, such as telecommunications, banking, gaming, retail, and insurance.

    Connect to Amazon MSK Serverless from your on-premises network

    Post Syndicated from Masudur Rahaman Sayem original https://aws.amazon.com/blogs/big-data/connect-to-amazon-msk-serverless-from-your-on-premises-network/

    Amazon Managed Streaming for Apache Kafka (Amazon MSK) is a fully managed, highly available, and secure Apache Kafka service. Amazon MSK reduces the work needed to set up, scale, and manage Apache Kafka in production. With Amazon MSK, you can create a cluster in minutes and start sending data.

    With Amazon MSK Serverless, you can run Apache Kafka without having to manage the underlying infrastructure. Amazon MSK will automatically provision, scale, and manage your Apache Kafka clusters, so you can focus on your applications without worrying about the operational overhead. Additionally, MSK Serverless offers fine-grained, pay-as-you-go pricing, making it a cost-effective option for organizations with unpredictable workloads.

    Connecting to MSK Serverless is easy. You can set up a serverless cluster using the API or AWS Management Console in minutes. MSK Serverless provides bootstrap information as a private DNS endpoint, allowing clients to connect to the serverless Apache Kafka cluster. A common use case of using MSK Serverless is an on-premises client that needs to process real-time data streams. However, the private DNS endpoint is only accessible from virtual private clouds (VPCs) that have been configured to connect and isn’t directly resolvable from an on-premises network. This can pose a challenge for on-premises clients to discover and connect to the MSK Serverless cluster.
    In this post, we guide you through a step-by-step process to connect your on-premises client to MSK Serverless, overcoming this challenge.

    Solution overview

    The following diagram illustrates the solution architecture.

    The flow of the solution is as follows:

    1. The DNS query for your MSK endpoint is routed to a locally configured on-premises DNS server.
    2. The on-premises DNS as configured performs conditional forwarding for kafka-serverless.REPLACE-MSK-SERVERLESS-REGION.amazonaws.com to an Amazon Route 53 inbound resolver endpoint IP address.
    3. The inbound resolver endpoint performs DNS resolution by forwarding the query to the private hosted zone that was created along with the MSK Serverless cluster.
    4. The IP addresses returned by the DNS query are the private IP addresses of the interface VPC endpoint, which allow your on-premises host to establish private connectivity over AWS VPN or AWS Direct Connect.
    5. The interface endpoint is a collection of one or more elastic network interfaces with a private IP address in your account that serves as an entry point for traffic destined to a MSK Serverless service.

    Note that at this time, this solution works only for MSK Serverless clusters with a single VPC.

    Prerequisites

    In this section, we discuss the prerequisite steps to complete in order to implement this solution.

    Establish network connectivity between on premises and the AWS Cloud

    To use MSK Serverless from your on-premises network, you need to establish a network connection between your on-premises environment and the VPC that you have set up for MSK Serverless. Various secure methods are available to connect your on-premises network to the AWS Cloud. Refer to Network-to-Amazon VPC connectivity options for more information.

    Create a security group for allowing inbound TCP/UDP connections from your on-premises network

    Create a security group with the following configurations on the same VPC that you configured for MSK Serverless:

    Inbound rule:

    • Source: [On-premises CIDR range]
    • Protocol: TCP/UDP
    • Port Range: 53

    Outbound rule: Leave it to default

    For more information, refer to Work with security groups.

    Update the MSK security group for inbound connections from your on-premises network

    To ensure that your MSK Serverless cluster can be accessed from your on-premises network, you need to adjust the cluster’s security group settings to allow incoming traffic from your network on TCP port 9098. Complete the following steps:

    1. On the Amazon MSK console, choose Clusters in the navigation pane.
    2. Navigate to your serverless MSK cluster’s properties.

    1. Choose the security group associated with your MSK cluster.

    Because MSK Serverless supports configuring multiple VPCs, make sure to choose the security group associated with the VPC that you configured for connecting from your on-premises network.

    1. To enable connections from your on-premises CIDR block to MSK Serverless, add an inbound rule that allows traffic on TCP port 9098 from your on-premises CIDR.

    This ensures that your on-premises network can communicate with MSK Serverless on the specified port.

    Configure a Route 53 inbound resolver endpoint

    MSK Serverless provides a DNS endpoint that serves as the starting point for an Apache Kafka client to connect to the cluster. However, this endpoint isn’t publicly discoverable and can only be accessed from within the configured VPC. To resolve the serverless DNS endpoint outside of your VPC, you can set up a Route 53 resolver endpoint. This allows you to access the endpoint securely by creating a hybrid cloud setup over VPN or Direct Connect.

    To configure the Route 53 resolver using the console, complete the following steps:

    1. On the Route 53 console, under Resolver in the navigation pane, choose Inbound endpoints.
    2. Choose Create inbound endpoint.

    1. For Endpoint name, enter the endpoint name.
    2. For VPC in the Region, choose the VPC where you configured MSK Serverless.
    3. For Security group for this endpoint, choose the security group that you created as a prerequisite for inbound TCP/UDP connections.

    The security group of the inbound resolver endpoint should allow traffic from the on-premises DNS Server IP address on TCP/UDP port 53.

    In the next step, you add your IP addresses, ensuring that the number of IP addresses matches the number of subnets in your MSK cluster.

    1. Choose the Availability Zones and subnets that are the same as your MSK Serverless network configuration.
    2. Select Use an IP address that is selected automatically.

    1. Choose Create inbound endpoint.

    1. Copy the inbound endpoint IP addresses.

    Configure the on-premises DNS server

    In this example, we use a Microsoft DNS server. To configure a conditional forwarder, complete the following steps:

    1. Open DNS Manager.
    2. Run the following command in the Run command window:
    dnsmgmt.msc
    1. Choose (right-click) Conditional Forwarders under the server of your choosing, then choose New Conditional Forwarder.


    In the next step, you enter kafka-serverless.REPLACE-MSK-SERVERLESS-REGION.amazonaws.com, using the IP address of Route 53 inbound resolver endpoints that you created earlier. You can find the MSK endpoint information by accessing the cluster’s client information. To learn more about getting client information, refer to Getting the bootstrap brokers for an Amazon MSK cluster.

    1. For DNS Domain, enter your endpoint name. For example, kafka-serverless.ap-southeast-2.amazonaws.com. Do not enter the entire endpoint name.
    2. Choose OK.

    Test the DNS resolution

    DNS (Domain Name System) uses TCP/UDP port 53. To test whether you can connect any of the Route 53 inbound endpoints, run the following command from your on-premises client:

    telnet Route53-INBOUND-ENDPOINT-IP 53

    For example: telnet 10.1.0.133 53

    The following is a sample output:

    Trying 10.1.0.133...
Connected to 10.1.0.133.
Escape character is '^]'.
Connection closed by foreign host.

    Run the following command to check whether you can connect with the MSK Serverless endpoint from your on-premises client. To get the MSK Serverless endpoint information, refer to Create an MSK Serverless cluster.

    dig MSK-SERVERLESS-ENDPOINT-REMOVE-PORT-NUMBER +short

    For example: dig boot-abcdc9.c3.kafka-serverless.ap-southeast-2.amazonaws.com +short

    The following is a sample output:

    vpce-0bcb06d53aab34111-vt8yzx2b.vpce-svc-05dc791a527abcd.ap-southeast-2.vpce.amazonaws.com.
10.1.1.185
10.1.0.191

    If the DNS resolution fails, check your network connectivity from on premises. For more information about troubleshooting connectivity issues, refer to How do I troubleshoot VPN tunnel connectivity to an Amazon VPC or Troubleshooting AWS Direct Connect.

    After you create a serverless MSK cluster, the service automatically creates an interface VPC endpoint for the cluster. You can use the dig command as shown above to retrieve the VPC endpoint ID and its associated IP address, which confirms that you are now able to connect to the MSK Serverless cluster from your on-premises environment.

    Test your Kafka client

    Once you complete the configuration of the Route 53 inbound resolver endpoint and on-premises DNS server, you can test your Kafka client from an on-premises network. For instructions, refer to Create a client machine. This documentation guides you through the necessary steps to set up your client machine and verify that it can successfully connect to your MSK cluster from your on-premises network.

    Conclusion

    MSK Serverless makes it easy for you to manage your data. You don’t have to worry about setting up and running your own Kafka cluster, which saves time and effort. In this post, we explored the option of on-premises connectivity with MSK Serverless and how it can greatly benefit organizations. By establishing this connection, you can gain access to a wide range of real-time analytics use case possibilities and unlock the full potential of your data.

    We encourage you to try on-premises connectivity with MSK serverless.

    About the Authors

    Masudur Rahaman Sayem is a Streaming Data Architect at AWS. He works with AWS customers globally to design and build data streaming architectures to solve real-world business problems. He specializes in optimizing solutions that use streaming data services and NoSQL. Sayem is very passionate about distributed computing.

    Akeef Khan is a Solutions Architect at Amazon Web Services. He helps SMB Greenfield customers adopt the cloud. Whilst being a generalist SA, Akeef is passionate about networking.

    What developers need to know about generative AI

    Post Syndicated from Damian Brady original https://github.blog/2023-04-07-what-developers-need-to-know-about-generative-ai/

    By now, you’ve heard of generative artificial intelligence (AI) tools like ChatGPT, DALL-E, and GitHub Copilot, among others. They’re gaining widespread interest thanks to the fact that they allow anyone to create content from email subject lines to code functions to artwork in a matter of moments.

    This potential to revolutionize content creation across various industries makes it important to understand what generative AI is, how it’s being used, and who it’s being used by. In this article, we’ll explore what generative AI is, how it works, some real-world applications, and how it’s already changing the way people (and developers) work.

    What is generative AI used for?

    You may have heard the buzz around new generative AI tools like ChatGPT or the new Bing, but there’s a lot more to generative AI than any one single framework, project, or application.

    Traditional AI systems are trained on large amounts of data to identify patterns, and they’re capable of performing specific tasks that can help people and organizations. But generative AI goes one step further by using complex systems and models to generate new, or novel, outputs in the form of an image, text, or audio based on natural language prompts.

    Generative AI models and applications can, for example, be used for:

    • Text generation. Text generation, as a field, with AI tools has been in development since the 1970s—but more recently, AI researchers have been able to train generative adversarial networks (GANs) to produce text that models human-like speech. A prime example is OpenAI’s application ChatGPT, which has been trained on thousands of texts, books, articles, and code repositories, and can respond with full answers to natural language prompts and questions.
    An example of text generation in ChatGPT
    An example of text generation in ChatGPT
    • Image generation. Generative AI models can be used to create new images with natural language prompts, which is one of the most popular techniques with current tools and applications. The goal with text-to-image generation is to create an image that accurately represents the content of a given prompt. For example, when we give the text prompt, “impressionist style oil painting of a Shiba Inu dog giving a tarot card reading,” to the popular AI image generator DALL-E 2 we get something that looks like this (and yes, it’s a gem):
    An AI-generated image from DALL-E 2 of a Shiba Inu dog giving a tarot card reading
    An AI-generated image from DALL-E 2 of a Shiba Inu dog giving a tarot card reading

    An example of a video created with a text prompt using diffusion models from [Imagen Video](https://imagen.research.google/).

    • Programming code generation. Rather than scouring the internet or developer community groups for help with code examples, generative AI models can be used to help generate new programming code with natural language prompts, complete partially written code with suggestions, or even translate code from one programming language to another. This is how, at a simple level, GitHub Copilot works: it uses OpenAI’sCodex model to offer code suggestions right from a developer’s editor. However, as you would with any software development tool, we encourage you to review generated code before merging into production.

    • Data generation. Creating new data—which is called synthetic data—and augmenting existing data sets is another common use case for generative AI. This involves generating new samples from an existing dataset to increase the dataset’s size and improve machine learning models trained on it, all while providing a layer of privacy since real user data is not being utilized to power models. Synthetic data generation provides a way to create useful, meaningful data for more than just ML training though—a number of self-driving car companies like Cruise and Waymo utilize AI-generated synthetic data for training perception systems to prepare vehicles for real-world situations while in operation.

    • Language translation. Natural-language understanding (NLU) models combined with generative AI have become increasingly popular to provide language translations on-the-fly. These types of tools help companies break language barriers and increase their scope of accessibility for customer bases by being able to provide things like support or documentation in their native language. Through complex, deep learning algorithms, generative AI is able to understand the context of a source text and linguistically construct those sentences in another language. This practice can also apply to coding languages, for example, translating a desired function from Python to Java.

    The bottom line: Even though generative AI is a relatively new technology, it’s already being used in consumer and business applications. The use cases, as well as the quantity of applications created with it, will continue evolving to meet more distinct and specific needs.

    How does generative AI work?

    Generative AI models work by using neural networks to identify patterns from large sets of data, then generate new and original data or content.

    But what are neural networks? In simple terms, they use interconnected nodes that are inspired by neurons in the human brain. These networks are the foundation of machine learning and deep learning models, which use a complex structure of algorithms to process large amounts of data such as text, code, or images. Training these neural networks involves adjusting the weights or parameters of the connections between neurons to minimize the difference between predicted and desired outputs, which allows the network to learn from mistakes and make more accurate predictions based on the data.

    Algorithms are a key component of machine learning and generative AI models. But beyond helping machines learn from data, algorithms are also used to optimize accuracy of outputs and make decisions, or recommendations, based on input data.

    While algorithms help automate these processes, building a generative AI model is incredibly complex due to the massive amounts of data and compute resources they require. People and organizations need large datasets to train these models, and generating high-quality data can be time-consuming and expensive.

    To restate the obvious, these models are complicated. Need proof? Here are some common generative AI models and how they work:

    • Large language models (LLM): LLMs are a type of machine learning model that process and generate natural language text. One of the most significant advancements in the development of large language models has been the availability of vast amounts of text data, such as books, websites, and social media posts. This data can be used to train models that are capable of predicting and generating natural language responses in a variety of contexts. As a result, large language models have multiple practical applications, such as virtual assistants, chatbots, or text generators, like ChatGPT.

    • Generative adversarial networks (GAN): GANs are one of the most used models for generative AI, and they employ two different neural networks. GANs consist of two different types of neural networks: a generator and a discriminator. The generator network generates new data, such as images or audio, from a random noise signal while the discriminator is trained to distinguish between real data from the training set and the data produced by the generator.

    During training, the generator tries to create data that can trick the discriminator network into thinking it’s real. This “adversarial” process will continue until the generator can produce data that is totally indistinguishable from real data in the training set. This process helps both networks improve at their respective tasks, which ultimately results in more realistic and higher-quality generated data.

    A diagram illustrating how a generative adversarial network works. Image [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/deed.en) האדם-החושב on wikipedia
    A diagram illustrating how a generative adversarial network works. Image [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/deed.en) האדם-החושב on wikipedia
    • Transformer-based models: A transformer-based model’s neural networks operate by learning context and meaning through tracking relationships of sequential data, which means these models are really good at natural language processing tasks like machine translation, language modeling, and answering questions. These models have been used in popular language models, such as GPT-4 (which stands for Generative Pre-trained Transformer 4), and have also been adapted for other such tasks that require the modeling of sequential data such as image recognition.
    • Variational autoencoder models (VAEs): These models are similar to GANs in that they work with two different neural networks: encoders and decoders. VAEs can take a large amount of data and compress it into a smaller representation, which can be used to create new data that is similar to the original data. VAEs are often used in image, video, and audio generation—and here’s a fun fact: you can train a VAE on datasets like CelebA, which contains over 200,000 images of celebrities, to create completely new portraits of people that don’t exist.
    The smile vector, a concept vector discovered by [Tom White](https://aiartists.org/tom-white) using VAEs trained on the CelebA dataset.
    The smile vector, a concept vector discovered by Tom White using VAEs trained on the CelebA dataset.

    The real-world applications of generative AI

    The impact of generative AI is quickly becoming apparent—but it’s still in its early days. Despite this, we’re already seeing a proliferation of applications, products, and open source projects that are using generative AI models to achieve specific outcomes for people and organizations (and yes, developers, too).

    Though generative AI is constantly evolving, it already has some solid real world applications. Here’s just a few of them:

    Coding

    New and seasoned developers alike can utilize generative AI to improve their coding processes. Generative AI coding tools can help automate some of the more repetitive tasks, like testing, as well as complete code or even generate brand new code. GitHub has its own AI-powered pair programmer, GitHub Copilot, which uses generative AI to provide developers with code suggestions. And GitHub also has announced GitHub Copilot X, which brings generative AI to more of the developer experience across the editor, pull requests, documentation, CLI, and more.

    Accessibility

    Generative AI has the potential to greatly impact and improve accessibility for folks with disabilities through a variety of modalities, such as speech-to-text transcription, text-to-speech audio generation, or assistive technologies. One of the most exciting facets of our GitHub Copilot tool is its voice-activated capabilities that allow developers with difficulties using a keyboard to code with their voice. By leveraging the power of generative AI, these types of tools are paving the way for a more inclusive and accessible future in technology.

    Gaming

    Generative AI can take gaming to the next level (get it? 😉) by generating new characters, storylines, design components, and more. Case in point: The developer behind the game, This Girl Does Not Exist, has said that every component of the game—from the storyline to the art and even the music—was generated entirely by AI. This use of generative AI can enable gaming studios to create new and exciting content for their users, all without increasing the developer workload, which frees them up to work on other aspects of the game, such as story development.

    Web design

    Designers can utilize generative AI tools to automate the design process and save significant time and resources, which allows for a more streamlined and efficient workflow. Additionally, incorporating these tools into the development process can lead to the creation of highly customized designs and logos, enhancing the overall user experience and engagement with the website or application. Generative AI tools can also be used to do some of the more tedious work, such as creating design layouts that are optimized and adaptable across devices. For example, designers can use tools like designs.ai to quickly generate logos, banners, or mockups for their websites.

    Microsoft and other industry players are increasingly utilizing generative AI models in search to create more personalized experiences. This includes query expansion, which generates relevant keywords to reduce the number of searches. So, rather than the search engine returning a list of links, generative AI can help these new and improved models return search results in the form of natural language responses. Bing now includes AI-powered features in partnership with OpenAI that provide answers to complex questions and allow users to ask follow-up questions in a chatbox for more refined responses.

    Healthcare

    Interest has emerged around the potential applications of generative AI in the healthcare field to improve disease detection and diagnosis, advance medical research, and accelerate progress in the pharmaceutical space. Potentially, generative AI could be used to analyze large amounts of data to simulate chemical structures and predict new compounds will be the most effective for new drug discoveries. NVIDIA Clara is one example of a generative AI model specifically designed for medical imaging and healthcare research. (Plus, Gartner suggests more than 30 percent of new pharmaceutical drugs and materials will be discovered via generative AI models by 2025.)

    Fun fact: Did you know that ChatGPT recently passed the US Medical Licensing exam without any intervention from clinicians?

    Marketing and advertising

    In marketing, content is king—and generative AI is making it easier than ever to quickly create large amounts of it. A number of companies, agencies, and creators are already turning to generative AI tools to create images for social posts or write captions, product descriptions, blog posts, email subject lines, and more. Generative AI can also help companies personalize ad experiences by creating custom, engaging content for individuals at speed. Writers, marketers, and creators can leverage tools like Jasper to generate copy, Surfer SEO to optimize organic search, or albert.ai to personalize digital advertising content.

    Art and design

    As we’ve seen above, the power of AI can be harnessed to create some incredible portraits in a matter of moments (re: the future-telling Shiba 🐕). Artists and designers alike are using these AI tools as a source of inspiration. For example, architects can quickly create 3D models of objects or environments and artists can breathe new life into their portraits by using AI to apply different styles, like adding a Cubist style to their original image. Need proof? Designers are already starting to use AI image generators, such as Midjourney and Microsoft Designer, to create high-quality images by simply typing out Discord commands.

    Finance

    In a recent discussion about tech trends and how they’ll affect the finance sector, Michael Schrage, a research fellow at the MIT Sloan School Initiative on the Digital Economy, said, “I think, increasingly, we’re going to be seeing generative AI used for financial forecasts and scenario generation.” This is a likely path forward—generative AI can be used to analyze large amounts of data to detect fraud, manage risk, and inform decision making. And that has obvious applications in the financial services industry.

    Manufacturing

    Manufacturers are starting to turn to generative AI solutions to help with product design, quality control, and predictive maintenance. Generative AI can be used to analyze historical data to improve machine failure predictions and help manufacturers with maintenance planning. According to research conducted by Capgemini, more than half of European manufacturers are implementing some AI solutions (although so far, these aren’t generative AI solutions). This is largely because the sheer amount of manufacturing data is easier for machines to analyze at speed than humans.

    AI as a partner: Generative AI models and tools are narrow in focus, and work best at generating content, code, and images. In research at GitHub, we’ve found that GitHub Copilot helps developers code up to 55% faster, underscoring how generative AI models and tools can improve overall productivity and boost efficiency. Metrics like these show how generative AI tools are already changing how people and teams work—but they also underscore how these tools act as complement to human efforts.

    Take this with you

    Whether it’s creating visual assets for an ad campaign or augmenting medical images to help diagnose diseases, generative AI is helping us solve complex problems at speed. And the emergence of generative AI-based programming tools has revolutionized the way developers approach writing code.

    We know that developers want to design and write software quickly, and tools like GitHub Copilot are enabling them to access large datasets to write more efficient code and boost productivity. In fact, 96% of developers surveyed reported spending less time on repetitive tasks using GitHub Copilot, which in turn allowed 74% of them to focus on more rewarding work.

    While these models aren’t perfect yet, they’re getting better by the day—and that’s creating an exciting immediate future for developers and generative AI.

    Security updates for Friday

    Post Syndicated from original https://lwn.net/Articles/928559/

    Security updates have been issued by Mageia (ldb/samba, libapreq2, opencontainers-runc, peazip, python-cairosvg, stellarium, and zstd), Oracle (httpd and mod_http2, kernel, and nss), SUSE (conmon, go1.19, go1.20, libgit2, openssl-1_1, and openvswitch), and Ubuntu (emacs24).

    The collective thoughts of the interwebz