Tag Archives: furniture

Cabinet of Secret Documents from Australia

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2018/02/cabinet_of_secr.html

This story of leaked Australian government secrets is unlike any other I’ve heard:

It begins at a second-hand shop in Canberra, where ex-government furniture is sold off cheaply.

The deals can be even cheaper when the items in question are two heavy filing cabinets to which no-one can find the keys.

They were purchased for small change and sat unopened for some months until the locks were attacked with a drill.

Inside was the trove of documents now known as The Cabinet Files.

The thousands of pages reveal the inner workings of five separate governments and span nearly a decade.

Nearly all the files are classified, some as “top secret” or “AUSTEO”, which means they are to be seen by Australian eyes only.

Yes, that really happened. The person who bought and opened the file cabinets contacted the Australian Broadcasting Corp, who is now publishing a bunch of it.

There’s lots of interesting (and embarassing) stuff in the documents, although most of it is local politics. I am more interested in the government’s reaction to the incident: they’re pushing for a law making it illegal for the press to publish government secrets it received through unofficial channels.

“The one thing I would point out about the legislation that does concern me particularly is that classified information is an element of the offence,” he said.

“That is to say, if you’ve got a filing cabinet that is full of classified information … that means all the Crown has to prove if they’re prosecuting you is that it is classified ­ nothing else.

“They don’t have to prove that you knew it was classified, so knowledge is beside the point.”

[…]

Many groups have raised concerns, including media organisations who say they unfairly target journalists trying to do their job.

But really anyone could be prosecuted just for possessing classified information, regardless of whether they know about it.

That might include, for instance, if you stumbled across a folder of secret files in a regular skip bin while walking home and handed it over to a journalist.

This illustrates a fundamental misunderstanding of the threat. The Australian Broadcasting Corp gets their funding from the government, and was very restrained in what they published. They waited months before publishing as they coordinated with the Australian government. They allowed the government to secure the files, and then returned them. From the government’s perspective, they were the best possible media outlet to receive this information. If the government makes it illegal for the Australian press to publish this sort of material, the next time it will be sent to the BBC, the Guardian, the New York Times, or Wikileaks. And since people no longer read their news from newspapers sold in stores but on the Internet, the result will be just as many people reading the stories with far fewer redactions.

The proposed law is older than this leak, but the leak is giving it new life. The Australian opposition party is being cagey on whether they will support the law. They don’t want to appear weak on national security, so I’m not optimistic.

EDITED TO ADD (2/8): The Australian government backed down on that new security law.

EDITED TO ADD (2/13): Excellent political cartoon.

Progressing from tech to leadership

Post Syndicated from Michal Zalewski original http://lcamtuf.blogspot.com/2018/02/on-leadership.html

I’ve been a technical person all my life. I started doing vulnerability research in the late 1990s – and even today, when I’m not fiddling with CNC-machined robots or making furniture, I’m probably clobbering together a fuzzer or writing a book about browser protocols and APIs. In other words, I’m a geek at heart.

My career is a different story. Over the past two decades and a change, I went from writing CGI scripts and setting up WAN routers for a chain of shopping malls, to doing pentests for institutional customers, to designing a series of network monitoring platforms and handling incident response for a big telco, to building and running the product security org for one of the largest companies in the world. It’s been an interesting ride – and now that I’m on the hook for the well-being of about 100 folks across more than a dozen subteams around the world, I’ve been thinking a bit about the lessons learned along the way.

Of course, I’m a bit hesitant to write such a post: sometimes, your efforts pan out not because of your approach, but despite it – and it’s possible to draw precisely the wrong conclusions from such anecdotes. Still, I’m very proud of the culture we’ve created and the caliber of folks working on our team. It happened through the work of quite a few talented tech leads and managers even before my time, but it did not happen by accident – so I figured that my observations may be useful for some, as long as they are taken with a grain of salt.

But first, let me start on a somewhat somber note: what nobody tells you is that one’s level on the leadership ladder tends to be inversely correlated with several measures of happiness. The reason is fairly simple: as you get more senior, a growing number of people will come to you expecting you to solve increasingly fuzzy and challenging problems – and you will no longer be patted on the back for doing so. This should not scare you away from such opportunities, but it definitely calls for a particular mindset: your motivation must come from within. Look beyond the fight-of-the-day; find satisfaction in seeing how far your teams have come over the years.

With that out of the way, here’s a collection of notes, loosely organized into three major themes.

The curse of a techie leader

Perhaps the most interesting observation I have is that for a person coming from a technical background, building a healthy team is first and foremost about the subtle art of letting go.

There is a natural urge to stay involved in any project you’ve started or helped improve; after all, it’s your baby: you’re familiar with all the nuts and bolts, and nobody else can do this job as well as you. But as your sphere of influence grows, this becomes a choke point: there are only so many things you could be doing at once. Just as importantly, the project-hoarding behavior robs more junior folks of the ability to take on new responsibilities and bring their own ideas to life. In other words, when done properly, delegation is not just about freeing up your plate; it’s also about empowerment and about signalling trust.

Of course, when you hand your project over to somebody else, the new owner will initially be slower and more clumsy than you; but if you pick the new leads wisely, give them the right tools and the right incentives, and don’t make them deathly afraid of messing up, they will soon excel at their new jobs – and be grateful for the opportunity.

A related affliction of many accomplished techies is the conviction that they know the answers to every question even tangentially related to their domain of expertise; that belief is coupled with a burning desire to have the last word in every debate. When practiced in moderation, this behavior is fine among peers – but for a leader, one of the most important skills to learn is knowing when to keep your mouth shut: people learn a lot better by experimenting and making small mistakes than by being schooled by their boss, and they often try to read into your passing remarks. Don’t run an authoritarian camp focused on total risk aversion or perfectly efficient resource management; just set reasonable boundaries and exit conditions for experiments so that they don’t spiral out of control – and be amazed by the results every now and then.

Death by planning

When nothing is on fire, it’s easy to get preoccupied with maintaining the status quo. If your current headcount or budget request lists all the same projects as last year’s, or if you ever find yourself ending an argument by deferring to a policy or a process document, it’s probably a sign that you’re getting complacent. In security, complacency usually ends in tears – and when it doesn’t, it leads to burnout or boredom.

In my experience, your goal should be to develop a cadre of managers or tech leads capable of coming up with clever ideas, prioritizing them among themselves, and seeing them to completion without your day-to-day involvement. In your spare time, make it your mission to challenge them to stay ahead of the curve. Ask your vendor security lead how they’d streamline their work if they had a 40% jump in the number of vendors but no extra headcount; ask your product security folks what’s the second line of defense or containment should your primary defenses fail. Help them get good ideas off the ground; set some mental success and failure criteria to be able to cut your losses if something does not pan out.

Of course, malfunctions happen even in the best-run teams; to spot trouble early on, instead of overzealous project tracking, I found it useful to encourage folks to run a data-driven org. I’d usually ask them to imagine that a brand new VP shows up in our office and, as his first order of business, asks “why do you have so many people here and how do I know they are doing the right things?”. Not everything in security can be quantified, but hard data can validate many of your assumptions – and will alert you to unseen issues early on.

When focusing on data, it’s important not to treat pie charts and spreadsheets as an art unto itself; if you run a security review process for your company, your CSAT scores are going to reach 100% if you just rubberstamp every launch request within ten minutes of receiving it. Make sure you’re asking the right questions; instead of “how satisfied are you with our process”, try “is your product better as a consequence of talking to us?”

Whenever things are not progressing as expected, it is a natural instinct to fall back to micromanagement, but it seldom truly cures the ill. It’s probable that your team disagrees with your vision or its feasibility – and that you’re either not listening to their feedback, or they don’t think you’d care. It’s good to assume that most of your employees are as smart or smarter than you; barking your orders at them more loudly or more frequently does not lead anyplace good. It’s good to listen to them and either present new facts or work with them on a plan you can all get behind.

In some circumstances, all that’s needed is honesty about the business trade-offs, so that your team feels like your “partner in crime”, not a victim of circumstance. For example, we’d tell our folks that by not falling behind on basic, unglamorous work, we earn the trust of our VPs and SVPs – and that this translates into the independence and the resources we need to pursue more ambitious ideas without being told what to do; it’s how we game the system, so to speak. Oh: leading by example is a pretty powerful tool at your disposal, too.

The human factor

I’ve come to appreciate that hiring decent folks who can get along with others is far more important than trying to recruit conference-circuit superstars. In fact, hiring superstars is a decidedly hit-and-miss affair: while certainly not a rule, there is a proportion of folks who put the maintenance of their celebrity status ahead of job responsibilities or the well-being of their peers.

For teams, one of the most powerful demotivators is a sense of unfairness and disempowerment. This is where tech-originating leaders can shine, because their teams usually feel that their bosses understand and can evaluate the merits of the work. But it also means you need to be decisive and actually solve problems for them, rather than just letting them vent. You will need to make unpopular decisions every now and then; in such cases, I think it’s important to move quickly, rather than prolonging the uncertainty – but it’s also important to sincerely listen to concerns, explain your reasoning, and be frank about the risks and trade-offs.

Whenever you see a clash of personalities on your team, you probably need to respond swiftly and decisively; being right should not justify being a bully. If you don’t react to repeated scuffles, your best people will probably start looking for other opportunities: it’s draining to put up with constant pie fights, no matter if the pies are thrown straight at you or if you just need to duck one every now and then.

More broadly, personality differences seem to be a much better predictor of conflict than any technical aspects underpinning a debate. As a boss, you need to identify such differences early on and come up with creative solutions. Sometimes, all you need is taking some badly-delivered but valid feedback and having a conversation with the other person, asking some questions that can help them reach the same conclusions without feeling that their worldview is under attack. Other times, the only path forward is making sure that some folks simply don’t run into each for a while.

Finally, dealing with low performers is a notoriously hard but important part of the game. Especially within large companies, there is always the temptation to just let it slide: sideline a struggling person and wait for them to either get over their issues or leave. But this sends an awful message to the rest of the team; for better or worse, fairness is important to most. Simply firing the low performers is seldom the best solution, though; successful recovery cases are what sets great managers apart from the average ones.

Oh, one more thought: people in leadership roles have their allegiance divided between the company and the people who depend on them. The obligation to the company is more formal, but the impact you have on your team is longer-lasting and more intimate. When the obligations to the employer and to your team collide in some way, make sure you can make the right call; it might be one of the the most consequential decisions you’ll ever make.

"Responsible encryption" fallacies

Post Syndicated from Robert Graham original http://blog.erratasec.com/2017/10/responsible-encryption-fallacies.html

Deputy Attorney General Rod Rosenstein gave a speech recently calling for “Responsible Encryption” (aka. “Crypto Backdoors”). It’s full of dangerous ideas that need to be debunked.

The importance of law enforcement

The first third of the speech talks about the importance of law enforcement, as if it’s the only thing standing between us and chaos. It cites the 2016 Mirai attacks as an example of the chaos that will only get worse without stricter law enforcement.

But the Mira case demonstrated the opposite, how law enforcement is not needed. They made no arrests in the case. A year later, they still haven’t a clue who did it.

Conversely, we technologists have fixed the major infrastructure issues. Specifically, those affected by the DNS outage have moved to multiple DNS providers, including a high-capacity DNS provider like Google and Amazon who can handle such large attacks easily.

In other words, we the people fixed the major Mirai problem, and law-enforcement didn’t.

Moreover, instead being a solution to cyber threats, law enforcement has become a threat itself. The DNC didn’t have the FBI investigate the attacks from Russia likely because they didn’t want the FBI reading all their files, finding wrongdoing by the DNC. It’s not that they did anything actually wrong, but it’s more like that famous quote from Richelieu “Give me six words written by the most honest of men and I’ll find something to hang him by”. Give all your internal emails over to the FBI and I’m certain they’ll find something to hang you by, if they want.
Or consider the case of Andrew Auernheimer. He found AT&T’s website made public user accounts of the first iPad, so he copied some down and posted them to a news site. AT&T had denied the problem, so making the problem public was the only way to force them to fix it. Such access to the website was legal, because AT&T had made the data public. However, prosecutors disagreed. In order to protect the powerful, they twisted and perverted the law to put Auernheimer in jail.

It’s not that law enforcement is bad, it’s that it’s not the unalloyed good Rosenstein imagines. When law enforcement becomes the thing Rosenstein describes, it means we live in a police state.

Where law enforcement can’t go

Rosenstein repeats the frequent claim in the encryption debate:

Our society has never had a system where evidence of criminal wrongdoing was totally impervious to detection

Of course our society has places “impervious to detection”, protected by both legal and natural barriers.

An example of a legal barrier is how spouses can’t be forced to testify against each other. This barrier is impervious.

A better example, though, is how so much of government, intelligence, the military, and law enforcement itself is impervious. If prosecutors could gather evidence everywhere, then why isn’t Rosenstein prosecuting those guilty of CIA torture?

Oh, you say, government is a special exception. If that were the case, then why did Rosenstein dedicate a precious third of his speech discussing the “rule of law” and how it applies to everyone, “protecting people from abuse by the government”. It obviously doesn’t, there’s one rule of government and a different rule for the people, and the rule for government means there’s lots of places law enforcement can’t go to gather evidence.

Likewise, the crypto backdoor Rosenstein is demanding for citizens doesn’t apply to the President, Congress, the NSA, the Army, or Rosenstein himself.

Then there are the natural barriers. The police can’t read your mind. They can only get the evidence that is there, like partial fingerprints, which are far less reliable than full fingerprints. They can’t go backwards in time.

I mention this because encryption is a natural barrier. It’s their job to overcome this barrier if they can, to crack crypto and so forth. It’s not our job to do it for them.

It’s like the camera that increasingly comes with TVs for video conferencing, or the microphone on Alexa-style devices that are always recording. This suddenly creates evidence that the police want our help in gathering, such as having the camera turned on all the time, recording to disk, in case the police later gets a warrant, to peer backward in time what happened in our living rooms. The “nothing is impervious” argument applies here as well. And it’s equally bogus here. By not helping police by not recording our activities, we aren’t somehow breaking some long standing tradit

And this is the scary part. It’s not that we are breaking some ancient tradition that there’s no place the police can’t go (with a warrant). Instead, crypto backdoors breaking the tradition that never before have I been forced to help them eavesdrop on me, even before I’m a suspect, even before any crime has been committed. Sure, laws like CALEA force the phone companies to help the police against wrongdoers — but here Rosenstein is insisting I help the police against myself.

Balance between privacy and public safety

Rosenstein repeats the frequent claim that encryption upsets the balance between privacy/safety:

Warrant-proof encryption defeats the constitutional balance by elevating privacy above public safety.

This is laughable, because technology has swung the balance alarmingly in favor of law enforcement. Far from “Going Dark” as his side claims, the problem we are confronted with is “Going Light”, where the police state monitors our every action.

You are surrounded by recording devices. If you walk down the street in town, outdoor surveillance cameras feed police facial recognition systems. If you drive, automated license plate readers can track your route. If you make a phone call or use a credit card, the police get a record of the transaction. If you stay in a hotel, they demand your ID, for law enforcement purposes.

And that’s their stuff, which is nothing compared to your stuff. You are never far from a recording device you own, such as your mobile phone, TV, Alexa/Siri/OkGoogle device, laptop. Modern cars from the last few years increasingly have always-on cell connections and data recorders that record your every action (and location).

Even if you hike out into the country, when you get back, the FBI can subpoena your GPS device to track down your hidden weapon’s cache, or grab the photos from your camera.

And this is all offline. So much of what we do is now online. Of the photographs you own, fewer than 1% are printed out, the rest are on your computer or backed up to the cloud.

Your phone is also a GPS recorder of your exact position all the time, which if the government wins the Carpenter case, they police can grab without a warrant. Tagging all citizens with a recording device of their position is not “balance” but the premise for a novel more dystopic than 1984.

If suspected of a crime, which would you rather the police searched? Your person, houses, papers, and physical effects? Or your mobile phone, computer, email, and online/cloud accounts?

The balance of privacy and safety has swung so far in favor of law enforcement that rather than debating whether they should have crypto backdoors, we should be debating how to add more privacy protections.

“But it’s not conclusive”

Rosenstein defends the “going light” (“Golden Age of Surveillance”) by pointing out it’s not always enough for conviction. Nothing gives a conviction better than a person’s own words admitting to the crime that were captured by surveillance. This other data, while copious, often fails to convince a jury beyond a reasonable doubt.
This is nonsense. Police got along well enough before the digital age, before such widespread messaging. They solved terrorist and child abduction cases just fine in the 1980s. Sure, somebody’s GPS location isn’t by itself enough — until you go there and find all the buried bodies, which leads to a conviction. “Going dark” imagines that somehow, the evidence they’ve been gathering for centuries is going away. It isn’t. It’s still here, and matches up with even more digital evidence.
Conversely, a person’s own words are not as conclusive as you think. There’s always missing context. We quickly get back to the Richelieu “six words” problem, where captured communications are twisted to convict people, with defense lawyers trying to untwist them.

Rosenstein’s claim may be true, that a lot of criminals will go free because the other electronic data isn’t convincing enough. But I’d need to see that claim backed up with hard studies, not thrown out for emotional impact.

Terrorists and child molesters

You can always tell the lack of seriousness of law enforcement when they bring up terrorists and child molesters.
To be fair, sometimes we do need to talk about terrorists. There are things unique to terrorism where me may need to give government explicit powers to address those unique concerns. For example, the NSA buys mobile phone 0day exploits in order to hack terrorist leaders in tribal areas. This is a good thing.
But when terrorists use encryption the same way everyone else does, then it’s not a unique reason to sacrifice our freedoms to give the police extra powers. Either it’s a good idea for all crimes or no crimes — there’s nothing particular about terrorism that makes it an exceptional crime. Dead people are dead. Any rational view of the problem relegates terrorism to be a minor problem. More citizens have died since September 8, 2001 from their own furniture than from terrorism. According to studies, the hot water from the tap is more of a threat to you than terrorists.
Yes, government should do what they can to protect us from terrorists, but no, it’s not so bad of a threat that requires the imposition of a military/police state. When people use terrorism to justify their actions, it’s because they trying to form a military/police state.
A similar argument works with child porn. Here’s the thing: the pervs aren’t exchanging child porn using the services Rosenstein wants to backdoor, like Apple’s Facetime or Facebook’s WhatsApp. Instead, they are exchanging child porn using custom services they build themselves.
Again, I’m (mostly) on the side of the FBI. I support their idea of buying 0day exploits in order to hack the web browsers of visitors to the secret “PlayPen” site. This is something that’s narrow to this problem and doesn’t endanger the innocent. On the other hand, their calls for crypto backdoors endangers the innocent while doing effectively nothing to address child porn.
Terrorists and child molesters are a clichéd, non-serious excuse to appeal to our emotions to give up our rights. We should not give in to such emotions.

Definition of “backdoor”

Rosenstein claims that we shouldn’t call backdoors “backdoors”:

No one calls any of those functions [like key recovery] a “back door.”  In fact, those capabilities are marketed and sought out by many users.

He’s partly right in that we rarely refer to PGP’s key escrow feature as a “backdoor”.

But that’s because the term “backdoor” refers less to how it’s done and more to who is doing it. If I set up a recovery password with Apple, I’m the one doing it to myself, so we don’t call it a backdoor. If it’s the police, spies, hackers, or criminals, then we call it a “backdoor” — even it’s identical technology.

Wikipedia uses the key escrow feature of the 1990s Clipper Chip as a prime example of what everyone means by “backdoor“. By “no one”, Rosenstein is including Wikipedia, which is obviously incorrect.

Though in truth, it’s not going to be the same technology. The needs of law enforcement are different than my personal key escrow/backup needs. In particular, there are unsolvable problems, such as a backdoor that works for the “legitimate” law enforcement in the United States but not for the “illegitimate” police states like Russia and China.

I feel for Rosenstein, because the term “backdoor” does have a pejorative connotation, which can be considered unfair. But that’s like saying the word “murder” is a pejorative term for killing people, or “torture” is a pejorative term for torture. The bad connotation exists because we don’t like government surveillance. I mean, honestly calling this feature “government surveillance feature” is likewise pejorative, and likewise exactly what it is that we are talking about.

Providers

Rosenstein focuses his arguments on “providers”, like Snapchat or Apple. But this isn’t the question.

The question is whether a “provider” like Telegram, a Russian company beyond US law, provides this feature. Or, by extension, whether individuals should be free to install whatever software they want, regardless of provider.

Telegram is a Russian company that provides end-to-end encryption. Anybody can download their software in order to communicate so that American law enforcement can’t eavesdrop. They aren’t going to put in a backdoor for the U.S. If we succeed in putting backdoors in Apple and WhatsApp, all this means is that criminals are going to install Telegram.

If the, for some reason, the US is able to convince all such providers (including Telegram) to install a backdoor, then it still doesn’t solve the problem, as uses can just build their own end-to-end encryption app that has no provider. It’s like email: some use the major providers like GMail, others setup their own email server.

Ultimately, this means that any law mandating “crypto backdoors” is going to target users not providers. Rosenstein tries to make a comparison with what plain-old telephone companies have to do under old laws like CALEA, but that’s not what’s happening here. Instead, for such rules to have any effect, they have to punish users for what they install, not providers.

This continues the argument I made above. Government backdoors is not something that forces Internet services to eavesdrop on us — it forces us to help the government spy on ourselves.
Rosenstein tries to address this by pointing out that it’s still a win if major providers like Apple and Facetime are forced to add backdoors, because they are the most popular, and some terrorists/criminals won’t move to alternate platforms. This is false. People with good intentions, who are unfairly targeted by a police state, the ones where police abuse is rampant, are the ones who use the backdoored products. Those with bad intentions, who know they are guilty, will move to the safe products. Indeed, Telegram is already popular among terrorists because they believe American services are already all backdoored. 
Rosenstein is essentially demanding the innocent get backdoored while the guilty don’t. This seems backwards. This is backwards.

Apple is morally weak

The reason I’m writing this post is because Rosenstein makes a few claims that cannot be ignored. One of them is how he describes Apple’s response to government insistence on weakening encryption doing the opposite, strengthening encryption. He reasons this happens because:

Of course they [Apple] do. They are in the business of selling products and making money. 

We [the DoJ] use a different measure of success. We are in the business of preventing crime and saving lives. 

He swells in importance. His condescending tone ennobles himself while debasing others. But this isn’t how things work. He’s not some white knight above the peasantry, protecting us. He’s a beat cop, a civil servant, who serves us.

A better phrasing would have been:

They are in the business of giving customers what they want.

We are in the business of giving voters what they want.

Both sides are doing the same, giving people what they want. Yes, voters want safety, but they also want privacy. Rosenstein imagines that he’s free to ignore our demands for privacy as long has he’s fulfilling his duty to protect us. He has explicitly rejected what people want, “we use a different measure of success”. He imagines it’s his job to tell us where the balance between privacy and safety lies. That’s not his job, that’s our job. We, the people (and our representatives), make that decision, and it’s his job is to do what he’s told. His measure of success is how well he fulfills our wishes, not how well he satisfies his imagined criteria.

That’s why those of us on this side of the debate doubt the good intentions of those like Rosenstein. He criticizes Apple for wanting to protect our rights/freedoms, and declare they measure success differently.

They are willing to be vile

Rosenstein makes this argument:

Companies are willing to make accommodations when required by the government. Recent media reports suggest that a major American technology company developed a tool to suppress online posts in certain geographic areas in order to embrace a foreign government’s censorship policies. 

Let me translate this for you:

Companies are willing to acquiesce to vile requests made by police-states. Therefore, they should acquiesce to our vile police-state requests.

It’s Rosenstein who is admitting here is that his requests are those of a police-state.

Constitutional Rights

Rosenstein says:

There is no constitutional right to sell warrant-proof encryption.

Maybe. It’s something the courts will have to decide. There are many 1st, 2nd, 3rd, 4th, and 5th Amendment issues here.
The reason we have the Bill of Rights is because of the abuses of the British Government. For example, they quartered troops in our homes, as a way of punishing us, and as a way of forcing us to help in our own oppression. The troops weren’t there to defend us against the French, but to defend us against ourselves, to shoot us if we got out of line.

And that’s what crypto backdoors do. We are forced to be agents of our own oppression. The principles enumerated by Rosenstein apply to a wide range of even additional surveillance. With little change to his speech, it can equally argue why the constant TV video surveillance from 1984 should be made law.

Let’s go back and look at Apple. It is not some base company exploiting consumers for profit. Apple doesn’t have guns, they cannot make people buy their product. If Apple doesn’t provide customers what they want, then customers vote with their feet, and go buy an Android phone. Apple isn’t providing encryption/security in order to make a profit — it’s giving customers what they want in order to stay in business.
Conversely, if we citizens don’t like what the government does, tough luck, they’ve got the guns to enforce their edicts. We can’t easily vote with our feet and walk to another country. A “democracy” is far less democratic than capitalism. Apple is a minority, selling phones to 45% of the population, and that’s fine, the minority get the phones they want. In a Democracy, where citizens vote on the issue, those 45% are screwed, as the 55% impose their will unwanted onto the remainder.

That’s why we have the Bill of Rights, to protect the 49% against abuse by the 51%. Regardless whether the Supreme Court agrees the current Constitution, it is the sort right that might exist regardless of what the Constitution says. 

Obliged to speak the truth

Here is the another part of his speech that I feel cannot be ignored. We have to discuss this:

Those of us who swear to protect the rule of law have a different motivation.  We are obliged to speak the truth.

The truth is that “going dark” threatens to disable law enforcement and enable criminals and terrorists to operate with impunity.

This is not true. Sure, he’s obliged to say the absolute truth, in court. He’s also obliged to be truthful in general about facts in his personal life, such as not lying on his tax return (the sort of thing that can get lawyers disbarred).

But he’s not obliged to tell his spouse his honest opinion whether that new outfit makes them look fat. Likewise, Rosenstein knows his opinion on public policy doesn’t fall into this category. He can say with impunity that either global warming doesn’t exist, or that it’ll cause a biblical deluge within 5 years. Both are factually untrue, but it’s not going to get him fired.

And this particular claim is also exaggerated bunk. While everyone agrees encryption makes law enforcement’s job harder than with backdoors, nobody honestly believes it can “disable” law enforcement. While everyone agrees that encryption helps terrorists, nobody believes it can enable them to act with “impunity”.

I feel bad here. It’s a terrible thing to question your opponent’s character this way. But Rosenstein made this unavoidable when he clearly, with no ambiguity, put his integrity as Deputy Attorney General on the line behind the statement that “going dark threatens to disable law enforcement and enable criminals and terrorists to operate with impunity”. I feel it’s a bald face lie, but you don’t need to take my word for it. Read his own words yourself and judge his integrity.

Conclusion

Rosenstein’s speech includes repeated references to ideas like “oath”, “honor”, and “duty”. It reminds me of Col. Jessup’s speech in the movie “A Few Good Men”.

If you’ll recall, it was rousing speech, “you want me on that wall” and “you use words like honor as a punchline”. Of course, since he was violating his oath and sending two privates to death row in order to avoid being held accountable, it was Jessup himself who was crapping on the concepts of “honor”, “oath”, and “duty”.

And so is Rosenstein. He imagines himself on that wall, doing albeit terrible things, justified by his duty to protect citizens. He imagines that it’s he who is honorable, while the rest of us not, even has he utters bald faced lies to further his own power and authority.

We activists oppose crypto backdoors not because we lack honor, or because we are criminals, or because we support terrorists and child molesters. It’s because we value privacy and government officials who get corrupted by power. It’s not that we fear Trump becoming a dictator, it’s that we fear bureaucrats at Rosenstein’s level becoming drunk on authority — which Rosenstein demonstrably has. His speech is a long train of corrupt ideas pursuing the same object of despotism — a despotism we oppose.

In other words, we oppose crypto backdoors because it’s not a tool of law enforcement, but a tool of despotism.

Encased in amber: meet the epoxy-embedded Pi

Post Syndicated from Janina Ander original https://www.raspberrypi.org/blog/epoxy-pi-resin-io/

The maker of one of our favourite projects from this year’s Maker Faire Bay Area took the idea of an ’embedded device’ and ran with it: Ronald McCollam has created a wireless, completely epoxy-encased Pi build – screen included!

Resin.io in resin epoxy-encased Raspberry Pi

*cue epic music theme* “Welcome…to resin in resin.”

Just encase…

Of course, this build is not meant to be a museum piece: Ronald embedded a Raspberry Pi 3 with built-in wireless LAN and Bluetooth to create a hands-on demonstration of the resin.io platform, for which he is a Solution Architect. Resin.io is useful for remotely controlling groups of Linux-based IoT devices. In this case, Ronald used it to connect to the encased Pi. And yes, he named his make Resin-in-resin – we salute you, sir!

resin.io in resin epoxy-encased Raspberry Pi

“Life uh…finds a way.”

Before he started the practical part of his project, he did his research to find a suitable resin. He found that epoxy types specifically designed for encasing electronics are very expensive. In the end, Ronald tried out a cheap type, usually employed to coat furniture, by encasing an LED. It worked perfectly, and he went ahead to use this resin for embedding the Pi.

Bubbleshooting epoxy

This was the first time Ronald had worked with resin, so he learned some essential things about casting. He advises other makers to mix the epoxy very, very slowly to minimize the formation of bubbles; to try their hands on some small-scale casting attempts first; and to make sure they’re using a large enough mold for casting. Another thing to keep in mind is that some components of the make will heat up and expand while the device is running.

His first version of an encased Pi was still connected to the outside world by its USB cable:

Ronald McCollam on Twitter

Updates don’t get more “hands off” than a Raspberry Pi encased in epoxy — @resin_io inside resin! Come ask me about it at @DockerCon!

Not satisfied with this, he went on to incorporate an inductive charging coil as a power source, so that the Pi could be totally insulated in epoxy. The Raspberry Pi Foundation’s Matt Richardson got a look the finished project at Maker Faire Bay Area:

MattRichardson🏳️‍🌈 on Twitter

If you’re at @makerfaire, you must check out what @resin_io is showing. A @Raspberry_Pi completely enclosed in resin. Completely wireless. https://t.co/djVjoLz3hI

MAGNETS!

The charging coil delivers enough power to keep the Pi running for several hours, but it doesn’t allow secure booting. After some head-scratching, Ronald came up with a cool solution to this problem: he added a battery and a magnetic reed switch. He explains:

[The] boot process is to use the magnetic switch to turn off the Pi, put it on the charger for a few minutes to allow the battery to charge up, then remove the magnet so the Pi boots.

Pi in resin controlled by resin.io

“God help us, we’re in the hands of engineers.”

He talks about his build on the resin.io blog, and has provided a detailed project log on Hackaday. For those of you who want to recreate this project at home, Ronald has even put together an Adafruit wishlist of the necessary components.

Does this resin-ate with you?

What’s especially great about Ronald’s posts is that they’re full of helpful tips about getting started with using epoxy resin in your digital making projects. So whether you’re keen to build your own wireless Pi, or just generally interested in embedding electronic components in resin, you’ll find his write-ups useful.

If you have experience in working with epoxy and electronic devices and want to share what you’ve learned, please do so in the comments!

The post Encased in amber: meet the epoxy-embedded Pi appeared first on Raspberry Pi.

Amazon Lightsail – The Power of AWS, the Simplicity of a VPS

Post Syndicated from Jeff Barr original https://aws.amazon.com/blogs/aws/amazon-lightsail-the-power-of-aws-the-simplicity-of-a-vps/

Some people like to assemble complex systems (houses, computers, or furniture) from parts. They relish the planning process, carefully researching each part and selecting those that give them the desired balance of power and flexibility. With planning out of the way, they enjoy the process of assembling the parts into a finished unit. Other people do not find this do-it-yourself (DIY) approach attractive or worthwhile, and are simply interested in getting to the results as quickly as possible without having to make too many decisions along the way.

Sound familiar?

I believe that this model applies to systems architecture and system building as well. Sometimes you want to take the time to hand-select individual AWS components (servers, storage, IP addresses, and so forth) and put them together on your own. At other times you simply need a system that is preconfigured and preassembled, and is ready to run your web applications with no system-building effort on your part.

In many cases, those seeking a preassembled system turned to a Virtual Private Server, or VPS. With a VPS, you are presented with a handful of options, each ready to run, and available to you for a predictable monthly fee.

While the VPS is a perfect getting-started vehicle, over time the environment can become constrained. At a certain point you may need to step outside the boundaries of the available plans as your needs grow, only to find that you have no options for incremental improvement, and are faced with the need to make a disruptive change. Or, you may find that your options for automated scaling or failover are limited, and that you need to set it all up yourself.

Introducing Amazon Lightsail
Today we are launching Amazon Lightsail. With a couple of clicks you can choose a configuration from a menu and launch a virtual machine preconfigured with SSD-based storage, DNS management, and a static IP address. You can launch your favorite operating system (Amazon Linux AMI, Ubuntu, CentOS, FreeBSD, or Debian), developer stack (LAMP, LEMP, MEAN, or Node.js), or application (Drupal, Joomla, Redmine, GitLab, and many others), with flat-rate pricing plans that start at $5 per month including a generous allowance for data transfer.

Here are the plans and the configurations:

You get the simplicity of a VPS, backed by the power, reliability, and security of AWS. As your needs grow, you will have the ability to smoothly step outside of the initial boundaries and connect to additional AWS database, messaging, and content distribution services.

All in all, Lightsail is the easiest way for you to get started on AWS and jumpstart your cloud projects, while giving you a smooth, clear path into the future.

A Quick Tour
Let’s take a quick tour of Amazon Lightsail! Each page of the Lightsail console includes a Quick Assist tab. You can click on it at any time in order to access context-sensitive documentation that will help you to get the most out of Lightsail:

I start at the main page. I have no instances or other resources at first:

I click on Create Instance to get moving. I choose my machine image (an App and an OS, or simply an OS) an instance plan, and give my instance a name, all on one page:

I can launch multiple instances, set up a configuration script, or specify an alternate SSH keypair if I’d like. I can also choose an Availability Zone. I’ll choose WordPress on the $10 plan, leave everything else as-is, and click on Create. It is up and running within seconds:

I can manage the instance by clicking on it:

My instance has a public IP address that I can open in my browser. WordPress is already installed, configured, and running:

I’ll need the WordPress password in order to finish setting it up. I click on Connect using SSH on the instance management page and I’m connected via a browser-based SSH terminal window without having to do any key management or install any browser plugins. The WordPress admin password is stored in file bitnami_application_password in the ~bitnami directory (the image below shows a made-up password):

You can bookmark the terminal window in order to be able to access it later with just a click or two.

I can manage my instance from the menu bar:

For example, I can access the performance metrics for my instance:

And I can manage my firewall settings:

I can capture the state of my instance by taking a Snapshot:

Later, I can restore the snapshot to a fresh instance:

I can also create static IP addresses and make use of domain names:

Advanced Lightsail – APIs and VPC Peering
Before I wrap up, let’s talk about a few of the more advanced features of Amazon Lightsail – APIs and VPC Peering.

As is almost always the case with AWS, there’s a full set of APIs behind all of the console functionality that we just reviewed. Here are just a few of the more interesting functions:

  • GetBundles – Get a list of the bundles (machine configurations).
  • CreateInstances – Create one or more Lightsail instances.
  • GetInstances – Get a list of all Lightsail instances.
  • GetInstance – Get information about a specific instance.
  • CreateInstanceSnapshot – Create a snapshot of an instance.
  • CreateInstanceFromSnapshot – Create an instance from a snapshot.

All of the Lightsail instances within an account run within a “shadow” VPC that is not visible in the AWS Management Console. If the code that you are running on your Lightsail instances needs access to other AWS resources, you can set up VPC peering between the shadow VPC and another one in your account, and create the resources therein. Click on Account (top right), scroll down to Advanced features, and check VPC peering:

You can now connect your Lightsail apps to other AWS resources that are running within a VPC.

Pricing and Availability
We are launching Amazon Lightsail today in the US East (Northern Virginia) Region, and plan to expand it to other regions in the near future.

Prices start at $5 per month.

Jeff;

AWS Week in Review – October 31, 2016

Post Syndicated from Jeff Barr original https://aws.amazon.com/blogs/aws/aws-week-in-review-october-31-2016/

Over 25 internal and external contributors helped out with pull requests and fresh content this week! Thank you all for your help and your support.

Monday

October 31

Tuesday

November 1

Wednesday

November 2

Thursday

November 3

Friday

November 4

Saturday

November 5

Sunday

November 6

New & Notable Open Source

New Customer Success Stories

  • Apposphere – Using AWS and bitfusion.io from the AWS Marketplace, Apposphere can scale 50 to 60 percent month-over-month while keeping customer satisfaction high. Based in Austin, Texas, the Apposphere mobile app delivers real-time leads from social media channels.
  • CADFEM – CADFEM uses AWS to make complex simulation software more accessible to smaller engineering firms, helping them compete with much larger ones. The firm specializes in simulation software and services for the engineering industry.
  • Mambu – Using AWS, Mambu helped one of its customers launch the United Kingdom’s first cloud-based bank, and the company is now on track for tenfold growth, giving it a competitive edge in the fast-growing fintech sector. Mambu is an all-in-one SaaS banking platform for managing credit and deposit products quickly, simply, and affordably.
  • Okta – Okta uses AWS to get new services into production in days instead of weeks. Okta creates products that use identity information to grant people access to applications on multiple devices at any time, while still enforcing strong security protections.
  • PayPlug – PayPlug is a startup created in 2013 that developed an online payment solution. It differentiates itself by the simplicity of its services and its ease of integration on e-commerce websites. PayPlug is a startup created in 2013 that developed an online payment solution. It differentiates itself by the simplicity of its services and its ease of integration on e-commerce websites
  • Rent-a-Center – Rent-a-Center is a leading renter of furniture, appliances, and electronics to customers in the United States, Canada, Puerto Rico, and Mexico. Rent-A-Center uses AWS to manage its new e-commerce website, scale to support a 1,000 percent spike in site traffic, and enable a DevOps approach.
  • UK Ministry of Justice – By going all in on the AWS Cloud, the UK Ministry of Justice (MoJ) can use technology to enhance the effectiveness and fairness of the services it provides to British citizens. The MoJ is a ministerial department of the UK government. MoJ had its own on-premises data center, but lacked the ability to change and adapt rapidly to the needs of its citizens. As it created more digital services, MoJ turned to AWS to automate, consolidate, and deliver constituent services.

New SlideShare Presentations

New YouTube Videos

Upcoming Events

Help Wanted

Stay tuned for next week! In the meantime, follow me on Twitter and subscribe to the RSS feed.

Sisyphus: the kinetic art table

Post Syndicated from Courtney Lentz original https://www.raspberrypi.org/blog/sisyphus-kinetic-art-table/

Surely if he had been given the opportunity, Sisyphus would have engineered a way out of his eternal punishment of rolling a boulder up a hill. It’s just too bad for him that Raspberry Pi wasn’t around to help. While it’s a far cry from his arduous task, the Pi has been used to power Bruce Shapiro’s Sisyphus, a continuous and ever-changing kinetic art piece that creates unique design patterns in sand using a small metal ball.

the-sisyphus-table-1-730x548

Sisyphus is truly mesmerising. We learned this first-hand: at Maker Faire New York earlier this month, it captured the attention of not only the Raspberry Pi crew, but also thousands of attendees throughout the weekend. Sisyphus momentarily drowned out the noise and action of the Faire.

You can think of Sisyphus as a cross between an Etch A Sketch and Spirograph, except this is no toy.

Under the table is a two-motor robot (the “Sisbot”) that moves a magnet which draws a steel ball through the sand. The motors are controlled by a small Raspberry Pi computer which plays a set of path files, much like a music player plays an MP3 file.

Sisyphus

Bruce is using Kickstarter in the hope of transitioning Sisyphus from what’s currently a large art installation exhibited around the world into a beautiful piece to be enjoyed in the home, as both furniture and art.

annmarie thomas on Twitter

Sisyphus- Stunning art/furniture kickstarter (fully funded in <a day) by friend Bruce Shapiro. https://t.co/ijxHQ0fYb5

Bruce says:

Of all works I made, Sisyphus stood out – it was my first CNC machine to break out of the studio/shop. No longer tasked with cutting materials to be used in making sculptures, it was the sculpture itself. It was also unique in another way – I wanted to live with it in my home. I’ve spent the last three years perfecting a home version that’s beautiful, user-friendly, near-silent, and that will run for years.

Like most great Maker Faire projects, it’s centred around a wonderful community. The collaboration and access to tools in Shapiro’s local makerspace helped develop the final design seen today. While Shapiro’s original makerspace has since closed its doors, Shapiro and his fellow members opened up what is now Nordeast Makers. It’s where the production for Sisyphus will take place.

Sisyphus

The Kickstarter products come in three styles: an end table, and two different coffee tables. You might want to find another place to display your coffee table books, though, so as to keep Sisyphus’s designs visible…

kickstarter-products

This Kickstarter won’t be running forever, so be sure to pledge if you love the sound of the Sisyphus.

The post Sisyphus: the kinetic art table appeared first on Raspberry Pi.

Five(ish) awesome RetroPie builds

Post Syndicated from Alex Bate original https://www.raspberrypi.org/blog/fiveish-awesome-retropie-builds/

If you’ve yet to hear about RetroPie, how’s it going living under that rock?

RetroPie, for the few who are unfamiliar, allows users to play retro video games on their Raspberry Pi or PC. From Alex Kidd to Ecco the Dolphin, Streets of Rage 2 to Cool Spot, nostalgia junkies can get their fill by flashing the RetroPie image to their Pi and plugging in their TV and a couple of USB controllers.

But for many, this simple setup is not enough. Alongside the RetroPie unit, many makers are building incredible cases and modifications to make their creation stand out from the rest.

Here’s five of what I believe to be some of the best RetroPie builds shared on social media:

1. Furniture Builds

If you don’t have the space for an arcade machine, why not incorporate RetroPie into your coffee table or desk?

This ‘Mid-century-ish Retro Games Table’ by Reddit user GuzziGuy fits a screen and custom-made controllers beneath a folding surface, allowing full use of the table when you’re not busy Space Raiding or Mario Karting.

GuzziGuy RetroPie Table

2. Arcade Cabinets

While the arcade cabinet at Pi Towers has seen better days (we have #LukeTheIntern working on it as I type), many of you makers are putting us to shame with your own builds. Whether it be a tabletop version or full 7ft cabinet, more and more RetroPie arcades are popping up, their builders desperate to replicate the sights of our gaming pasts.

One maker, YouTuber Bob Clagett, built his own RetroPie Arcade Cabinet from scratch, documenting the entire process on his channel.

With sensors that start the machine upon your approach, LED backlighting, and cartoon vinyl artwork of his family, it’s easy to see why this is a firm favourite.

Arcade Cabinet build – Part 3 // How-To

Check out how I made this fully custom arcade cabinet, powered by a Raspberry Pi, to play retro games! Subscribe to my channel: http://bit.ly/1k8msFr Get digital plans for this cabinet to build your own!

3. Handheld Gaming

If you’re looking for a more personal gaming experience, or if you simply want to see just how small you can make your build, you can’t go wrong with a handheld gaming console. With the release of the Raspberry Pi Zero, the ability to fit an entire RetroPie setup within the smallest of spaces has become somewhat of a social media maker challenge.

Chase Lambeth used an old Burger King toy and Pi Zero to create one of the smallest RetroPie Gameboys around… and it broke the internet in the process.

Mini Gameboy Chase Lambeth

4. Console Recycling

What better way to play a retro game than via a retro game console? And while I don’t condone pulling apart a working NES or MegaDrive, there’s no harm in cannibalising a deceased unit for the greater good, or using one of many 3D-printable designs to recreate a classic.

Here’s YouTuber DaftMike‘s entry into the RetroPie Hall of Fame: a mini-NES with NFC-enabled cartridges that autoplay when inserted.

Raspberry Pi Mini NES Classic Console

This is a demo of my Raspberry Pi ‘NES Classic’ build. You can see photos, more details and code here: http://www.daftmike.com/2016/07/NESPi.html Update video: https://youtu.be/M0hWhv1lw48 Update #2: https://youtu.be/hhYf5DPzLqg Electronics kits are now available for pre-order, details here: http://www.daftmike.com/p/nespi-electronics-kit.html Build Guide Update: https://youtu.be/8rFBWdRpufo Build Guide Part 1: https://youtu.be/8feZYk9HmYg Build Guide Part 2: https://youtu.be/vOz1-6GqTZc New case design files: http://www.thingiverse.com/thing:1727668 Better Snap Fit Cases!

5. Everything Else

I can’t create a list of RetroPie builds without mentioning the unusual creations that appear on our social media feeds from time to time. And while you may consider putting more than one example in #5 cheating, I say… well, I say pfft.

Example 1 – Sean (from SimpleCove)’s Retro Arcade

It felt wrong to include this within Arcade Cabinets as it’s not really a cabinet. Creating the entire thing from scratch using monitors, wood, and a lot of veneer, the end result could easily have travelled here from the 1940s.

Retro Arcade Cabinet Using A Raspberry Pi & RetroPie

I’ve wanted one of these raspberry pi/retro pi arcade systems for a while but wanted to make a special box to put it in that looked like an antique table top TV/radio. I feel the outcome of this project is exactly that.

Example 2 – the HackerHouse Portable Console… built-in controller… thing

The team at HackerHouse, along with many other makers, decided to incorporate the entire RetroPie build into the controller, allowing you to easily take your gaming system with you without the need for a separate console unit. Following on from the theme of their YouTube channel, they offer a complete tutorial on how to make the controller.

Make a Raspberry Pi Portable Arcade Console (with Retropie)

Find out how to make an easy portable arcade console (cabinet) using a Raspberry Pi. You can bring it anywhere, plug it into any tv, and play all your favorite classic ROMs. This arcade has 4 general buttons and a joystick, but you can also plug in any old usb enabled controller.

Example 3 – Zach’s PiCart

RetroPie inside a NES game cartridge… need I say more?

Pi Cart: a Raspberry Pi Retro Gaming Rig in an NES Cartridge

I put a Raspberry Pi Zero (and 2,400 vintage games) into an NES cartridge and it’s awesome. Powered by RetroPie. I also wrote a step-by-step guide on howchoo and a list of all the materials you’ll need to build your own: https://howchoo.com/g/mti0oge5nzk/pi-cart-a-raspberry-pi-retro-gaming-rig-in-an-nes-cartridge

Here’s a video to help you set up your own RetroPie. What games would you play first? And what other builds have caught your attention online?

The post Five(ish) awesome RetroPie builds appeared first on Raspberry Pi.

Amazon ECS Service Auto Scaling Enables Rent-A-Center SAP Hybris Solution

Post Syndicated from Chris Barclay original https://aws.amazon.com/blogs/compute/amazon-ecs-service-auto-scaling-enables-rent-a-center-sap-hybris-solution/

This is a guest post from Troy Washburn, Sr. DevOps Manager @ Rent-A-Center, Inc., and Ashay Chitnis, Flux7 architect.

—–

Rent-A-Center in their own words: Rent-A-Center owns and operates more than 3,000 rent-to-own retail stores for name-brand furniture, electronics, appliances and computers across the US, Canada, and Puerto Rico.

Rent-A-Center (RAC) wanted to roll out an ecommerce platform that would support the entire online shopping workflow using SAP’s Hybris platform. The goal was to implement a cloud-based solution with a cluster of Hybris servers which would cater to online web-based demand.

The challenge: to run the Hybris clusters in a microservices architecture. A microservices approach has several advantages including the ability for each service to scale up and down to meet fluctuating changes in demand independently. RAC also wanted to use Docker containers to package the application in a format that is easily portable and immutable. There were four types of containers necessary for the architecture. Each corresponded to a particular service:

1. Apache: Received requests from the external Elastic Load Balancing load balancer. Apache was used to set certain rewrite and proxy http rules.
2. Hybris: An external Tomcat was the frontend for the Hybris platform.
3. Solr Master: A product indexing service for quick lookup.
4. Solr Slave: Replication of master cache to directly serve product searches from Hybris.

To deploy the containers in a microservices architecture, RAC and AWS consultants at Flux7 started by launching Amazon ECS resources with AWS CloudFormation templates. Running containers on ECS requires the use of three primary resources: clusters, services, and task definitions. Each container refers to its task definition for the container properties, such as CPU and memory. And, each of the above services stored its container images in Amazon ECR repositories.

This post describes the architecture that we created and implemented.

Auto Scaling

At first glance, scaling on ECS can seem confusing. But the Flux7 philosophy is that complex systems only work when they are a combination of well-designed simple systems that break the problem down into smaller pieces. The key insight that helped us design our solution was understanding that there are two very different scaling operations happening. The first is the scaling up of individual tasks in each service and the second is the scaling up of the cluster of Amazon EC2 instances.

During implementation, Service Auto Scaling was released by the AWS team and so we researched how to implement task scaling into the existing solution. As we were implementing the solution through AWS CloudFormation, task scaling needed to be done the same way. However, the new scaling feature was not available for implementation through CloudFormation and so the natural course was to implement it using AWS Lambda–backed custom resources.

A corresponding Lambda function is implemented in Node.js 4.3, while automatic scaling happens by monitoring the CPUUtilization Amazon CloudWatch metric. The ECS policies below are registered with CloudWatch alarms that are triggered when specific thresholds are crossed. Similarly, by using the MemoryUtilization CloudWatch metric, ECS scaling can be made to scale in and out as well.

The Lambda function and CloudFormation custom resource JSON are available in the Flux7 GitHub repository: https://github.com/Flux7Labs/blog-code-samples/tree/master/2016-10-ecs-enables-rac-sap-hybris

Scaling ECS services and EC2 instances automatically

The key to understanding cluster scaling is to start by understanding the problem. We are no longer running a homogeneous workload in a simple environment. We have a cluster hosting a heterogeneous workload with different requirements and different demands on the system.

This clicked for us after we phrased the problem as, “Make sure the cluster has enough capacity to launch ‘x’ more instances of a task.” This led us to realize that we were no longer looking at an overall average resource utilization problem, but rather a discrete bin packing problem.

The problem is inherently more complex. (Anyone remember from algorithms class how the discrete Knapsack problem is NP-hard, but the continuous knapsack problem can easily be solved in polynomial time? Same thing.) So we have to check on each individual instance if a particular task can be scheduled on it, and if for any task we don’t cross the required capacity threshold, then we need to allocate more instance capacity.

To ensure that ECS scaling always has enough resources to scale out and has just enough resources after scaling in, it was necessary that the Auto Scaling group scales according to three criteria:

1. ECS task count in relation to the host EC2 instance count in a cluster
2. Memory reservation
3. CPU reservation

We implemented the first criteria for the Auto Scaling group. Instead of using the default scaling abilities, we set group scaling in and out using Lambda functions that were triggered periodically by a combination of the AWS::Lambda::Permission and an AWS::Events::Rule resources, as we wanted specific criteria for scaling.

The Lambda function is available in the Flux7 GitHub repository: https://github.com/Flux7Labs/blog-code-samples/tree/master/2016-10-ecs-enables-rac-sap-hybris

Future versions of this piece of code will incorporate the other two criteria along with the ability to use CloudWatch alarms to trigger scaling.

Conclusion

Using advanced ECS features like Service Auto Scaling in conjunction with Lambda to meet RAC’s business requirements, RAC and Flux7 were able to Dockerize SAP Hybris in production for the first time ever.

Further, ECS and CloudFormation give users the ability to implement robust solutions while still providing the ability to roll back in case of failures. With ECS as a backbone technology, RAC has been able to deploy a Hybris setup with automatic scaling, self-healing, one-click deployment, CI/CD, and PCI compliance consistent with the company’s latest technology guidelines and meeting the requirements of their newly-formed culture of DevOps and extreme agility.

If you have any questions or suggestions, please comment below.

Hot Startups on AWS – July 2016 – Depop, Nextdoor, Branch

Post Syndicated from Jeff Barr original https://aws.amazon.com/blogs/aws/hot-startups-on-aws-july-2016/

Today I would like to introduce a very special guest blogger! My daughter Tina is a Recruiting Coordinator for the AWS team and is making her professional blogging debut with today’s post.


Jeff;


It’s officially summer and it’s hot! Check out this month’s hot AWS-powered startups:

  • Depop – a social mobile marketplace for artists and friends to buy and sell products.
  • Nextdoor – building stronger and safer neighborhoods through technology.
  • Branch – provides free deep linking technology for mobile app developers to gain and retain users.

Depop (UK)
In 2011, Simon Beckerman and his brother, Daniel, set out to create a social, mobile marketplace that would make buying and selling from mobile a fun and interactive experience. The Depop founders recognized that the rise of m-commerce was changing the way that consumers wanted to communicate and interact with each other. Simon, who already ran PIG Magazine and the luxury eyewear brand RetroSuperFuture, wanted to create a space where artists and creatives like himself could share, buy and sell their possessions. After launching organically in Italy, Depop moved to Shoreditch, London in 2012 to establish its headquarter and has since grown considerably with offices in London, New York, and Milan.

With over 4 million users worldwide, Depop is growing and building a community of shop owners with a passion for fashion, music, art, vintage, and lifestyle pieces. The familiar and user-friendly interface allows users to follow, like, comment, and private message with other users and shop owners. Simply download the app (Android or iOS) and you are suddenly connected to millions of unique items ready for purchase. It’s not just clothes either – you can find home décor, vintage furniture, jewelry, and more. Filtering by location allows you to personalize your feed and shop locally for even more convenience. Buyers can scroll through an endless stream of items ready for purchase and have the option to either pick up in-person or have their items shipped directly to them. Selling items is just as easy – upload a photo, write a short description, set a price, and then list your product.

Depop chose AWS in order to move fast without needing a large operations team, following a DevOps approach. They use 12 distinct AWS services including Amazon S3 and Amazon CloudFront for image hosting, and Auto Scaling to deal with the unpredictable and fairly large changes in traffic throughout the day. Depop’s developers are able to support their own services in production without needing to call on a dedicated operations team.

Check out Depop’s Blog to keep up with the artists using the app!

Nextdoor (San Francisco)
Based in San Francisco, Nextdoor has helped more than 100,000 neighborhoods across the United States bring their communities closer together. In 2010, the founders of this startup were surprised to learn from a Pew research study that the majority of American adults knew only some (29%) or none (28%) of their neighbors by name. Recognizing an opportunity to bring back a sense of community to neighborhoods across the country, the idea for Nextdoor was born. Neighbors are using Nextdoor to ask questions, get to know one another, and exchange local advice and recommendations. For example, neighbors are able to help one another to:

  • Find trustworthy babysitters, plumbers, and dentists in the area.
  • Organize neighborhood events, such as garage sales and block parties.
  • Get assistance to find lost pets and missing packages.
  • Sell or give away items, like an old kitchen table or bike.
  • Report neighborhood crime and share safety concerns.

Nextdoor is also giving local agencies such as police and fire departments, and offices of emergency management the ability to connect with verified residents in their jurisdiction through a feature called Nextdoor for Public Agencies. This is incredibly beneficial for agencies to help residents with emergency preparedness, community engagement, crime prevention, and community policing. In his seminal work, Bowling Alone, Harvard Professor Robert Putnam learned that when social capital within a community is high, children do better in school, neighborhoods are safer, people prosper, the government is better, and people are happier and healthier overall. With a comprehensive list of helpful community guidelines, Nextdoor is creating stronger and safer neighborhoods with the power of technology. You can download the Nextdoor app for Android or iOS.

AWS  is the foundational infrastructure for both the online services in Nextdoor’s technology stack, and all of their offline data processing and analytics systems. Nextdoor uses over 25 different AWS services (Amazon EC2, Elastic Loud Balancing, Amazon Cloudfront, Amazon S3, Amazon DynamoDB, Amazon Redshift, and Amazon Kinesis to name a few) to quickly prototype, develop, and deploy new features for community members. Supporting millions of users in the US, Nextdoor runs their services across four AWS Regions worldwide, and has also recently expanded to Europe. In their own words, “Amazon makes it easy for us to flexibly grow our technology footprint with predictable costs in an automated fashion.”

Branch (Palo Alto)
The idea for Branch came in May 2014 when a group of Stanford business school graduates began working together to build and launch their own mobile app. They soon realized how challenging it was to grow their app, and saw that many of their friends were running into the same difficulties. The graduates saw the potential to create a deep linking platform to help apps get discovered, retain users, and grow exponentially. Branch reached its first million users within several months after its inception, and a little over a year later had climbed to one billion users and 5,000 apps. Companies such as Pinterest, Instacart, Mint, and Redfin are partnering with Branch to improve their user experience worldwide. Over 11,000 apps use the platform today.

As the number of smartphone users continues to increase, mobile apps are providing better user experiences, higher conversions, and better retention rates than the mobile web. The issue comes when mobile developers want to link users to the content they worked so hard to create – the transition between emails, ads, referrals, and more can often lead to broken experiences.

Mobile deep links allow users to share content that is within an app. Normal web links don’t work unless apps are downloaded on a device, and even then there is no standard way to find and share content as it is specific to every app. Branch allows content within apps to be shared just as they would be on the web. For example, imagine you are shopping for a fresh pair of shoes on the mobile web. You are ready to check out, but are prompted to download the store’s app to complete your purchase. Now that you’ve downloaded the app, you are brought back to the store’s homepage and need to restart your search from the beginning. With a Branch deep link, you instead would be linked directly back to checkout once you’ve installed the app, saving time and creating an overall better user experience.

Branch has grown exponentially over the past two years, and relies heavily on AWS to scale its infrastructure. Anticipating continued growth, Branch builds and maintains most of its infrastructure services with open source tools running on Amazon EC2 instances (Amazon API Gateway, Apache Kafka, Apache Zookeeper, Kubernetes, Redis, and Aerospike), and also use AWS services such as Elastic Load Balancing, Amazon CloudFront, Amazon Route 53, and Amazon RDS for PostgreSQL. These services allow Branch to maintain a 99.999% success rate on links with a latency of only 60 ms in the 99th percentile. To learn more about how they did this, read their recent blog post, Scaling to Billions of Requests a Day with AWS.

Tina Barr

One small step for Steph, one giant flap for makerkind

Post Syndicated from Steph Burton original https://www.raspberrypi.org/blog/one-small-step-for-steph/

I’m Steph, I’m quite new to the Raspberry Pi Foundation, and I’m very new to Raspberry Pis. Until quite recently, any mention of pie to me meant that good food was on the horizon – now my horizons are much broader. I’ve been part of the Programmes Team at Pi Towers for about 3 months, and I’ve only just picked my jaw up from the floor in awe of the creative geniuses around me. The things that I’ve seen are mindboggling and I began to wonder how they were created. Well, there was only one way to find out – make something myself.

Steph and her creation

The smile of a happy maker

The time had come for me to get started in the world of digital making. I’ve always been into arts and crafts, and I love to put my own touch and personality on my possessions; sewing buttons and colourful things on to clothes, revamping drawer knobs, applying découpage to any plain bits of furniture, and taking over the world with my glue gun. However, making something digital from scratch was a daunting prospect! I wasn’t going to let it scare me, though; I’ve dived out of a plane before and landed with a smile on my face.

So, supported by my team and with that ‘Friday feeling’, I took the plunge and transformed into a digital maker for the afternoon. I was presented with a DIY Gamer Kit, from Technology Will Save Us, as my first project. I opened the box of components and loaded up the online instructions, then I had to take a deep breath and compose myself as I read the word ‘solder’. I was very excited that I was going to need to solder, then realised I didn’t know how to do it. Rachel Rayns, my lovely desk buddy, gave me a soldering tutorial; now, I feel like I can solder most metal things in the world. I loved it.

Steph learns to solder

The soldering skills that earned an Instagram marriage proposal.

I continued the rest of my mission on my own, with the incentive of being able to play Snake at the end of it. I worked my way through the kit, identifying all of the digital ingredients and joining them together in the right places. I soldered push buttons, LEDs, a buzzer, resistors, and many other components to a PCB (printed circuit board). I was amazed at how quickly the parts grew into a device that looked very much like a Game Boy, and I was impressed to see how it matched the photo instructions – very useful!

Following the instructions, I added a brain to my device in the form of an Arduino, and an acrylic accessory to the front and back. This was a great way to protect my game; if I’m honest, I may have dropped it a few times whilst fumbling through the engineers’ tool stash. Luckily, nothing fell apart, which was a testament to my new-found soldering skills. After fixing the spacers, nuts and bolts in place, the only thing left to do was to connect a 9v battery to the game, and then run through the office waving it around when it powered up.

I had made the DIY Gamer Kit, and in that moment I wasn’t sure which thing made me smile the most:

  • How quickly I was able to put it together – even though I stopped to admire my work every 5 minutes
  • The fact that I could now play Snake
  • Knowing that, against all odds, I hadn’t burnt myself
  • The idea of going to make something else straight away

Once my smile had shrunk back down to normal size, I was calm enough to think about doing some coding. I’ve been told that code can be used to solve real life problems, and I certainly needed it when I uploaded the game ‘Flappy Bird’ on to my new game machine and couldn’t survive for longer than 2 seconds. My problem was that my bird was flying far too fast to control – it had to be hacked! Again, with the help of Rachel we hacked the game and adapted the code. I was then able to play Flappy Bird at a much more reasonable flying speed. My problems didn’t quite stop here, though, as I continued to fly my bird into wall after wall, ending the game prematurely. We hacked it some more, and now I’ll never see the words ‘Game Over’ again.

Rachel and Steph go through Coding 101

Coding 101

I’ve been inspired to be more of a digital maker, because I enjoyed every minute of my very first project. I hope that others may find the same inspiration from the amount of joy on my face in the picture below. Go forth and make something, and you too could be this happy.

The post One small step for Steph, one giant flap for makerkind appeared first on Raspberry Pi.

A Raspberry Pi + IKEA arcade table to make yourself

Post Syndicated from Helen Lynn original https://www.raspberrypi.org/blog/raspberry-pi-ikea-arcade-table-make-yourself/

Barely a month slips by at the moment without my ordering some new flat-packed goodies from IKEA. Our family, still gradually settling into the house we moved into just before our eldest was born, goes about its book-savouring, toy-categorising, craft-supply-hoarding life within a sturdy framework of TROFAST, EKBY and BESTÅ. The really great thing is that much of this furniture lends itself to modification, and spannerspencer‘s PIK3A Gaming Table, using a Raspberry Pi and the iconic LACK side table, is a wonderful example.

PIK3A gaming table - a glossy red IKEA LACK table with inlaid monitor, joystick and buttons

Shiny retrogaming loveliness

The build instructions over at element14 are generously illustrated with photographs, bringing this project within reach of people who don’t have a ton of experience, but are happy to chuck some time at it. (If I give this one a go, I’ll probably start by getting a couple of tables so that I have a back-up. The mods to the table don’t need any fancy tools – just a drill, a Stanley knife and a hole saw – but these are the steps at greatest risk of mistakes you can’t undo.) The tutorial takes you through everything from cutting the table so as to avoid too many repeat attempts, to mounting and wiring up the controls, to the code you need to run on the Arduino and how to upload it.

Cutting holes in an IKEA LACK table for buttons and other controls

Holes much neater than the ones I will cut

You can buy a new LACK table for £6 in the UK, although the nice red glossy version in the pictures will set you back a whole £2 more. A Raspberry Pi, an Arduino Leonardo, an old LCD monitor, some cheap computer speakers, a joystick, buttons, cables and connectors, and a power supply complete the bill of materials for this build. If you want to make it extra beautiful or simply catproof it, you can add a sheet of acrylic to protect the monitor, as spannerspencer has. He’s also included a panel mount USB port to make it easy to add USB peripherals later.

A cat standing on a PIK3A gaming table protected with a sheet of transparent acrylic

PIK3A, with added catproofing

The PIK3A Gaming Table went down a storm over at element14, and its successor, the PIK3A Mark II two-player gaming table (using a LACK TV bench) is proving pretty popular too. Give them a go!

The post A Raspberry Pi + IKEA arcade table to make yourself appeared first on Raspberry Pi.