Tag Archives: transportation

Ultrafast Motion-Planning Chip Could Make Autonomous Cars Safer

Post Syndicated from Evan Ackerman original https://spectrum.ieee.org/cars-that-think/transportation/self-driving/realtime-robotics-motion-planning-chip-autonomous-cars

Realtime Robotics’ motion-planning processor helps autonomous cars make better decisions

About two years ago, we covered a research project from Duke University that sped up motion planning for a tabletop robot arm by several orders of magnitude. The robot relied on a custom processor to do in milliseconds what normally takes seconds. The Duke researchers formed a company based on this tech called Realtime Robotics, and recently they’ve been focused on applying it to autonomous vehicles.

The reason that you should care about fast motion planning for autonomous vehicles is because motion planning encompasses the process by which the vehicle decides what it’s going to do next. Making this process faster doesn’t just mean that the vehicle can make decisions more quickly, but that it can make much better decisions as well—keeping you, and everyone around you, as safe as possible.

Smartphones Replace Fobs for Keyless Vehicle Entry

Post Syndicated from Lawrence Ulrich original https://spectrum.ieee.org/cars-that-think/transportation/advanced-cars/smartphones-replace-fobs-for-keyless-vehicle-entry

Lincoln and Amazon are the latest companies to harness smartphone apps for easy vehicle entry

Roll-up windows. Cigarette lighters. Physical ignition keys. All of these features have gone virtually extinct in modern automobiles. The quaint metal key gave way to transponder fobs, which led to “proximity keys” that don’t leave your pocket at all. Now, smartphones are becoming the new gatekeepers, as car companies roll out features that let drivers unlock and start their cars through an app.

Volvo began offering its subscription-based On Call service in 2016; it allows owners to use the company’s smartphone app to lock, unlock, and start their cars. With it, users can also remotely check vehicle fuel levels, receive service alerts, or send destinations to the onboard navigation system.

Tesla, which has never been shy about beta testing on its customers, then attempted to sell a car with no fob at all: The Model 3 sedan was initially available with only smartphone-based entry and ignition, and a backup RFID card. Beset by customer complaints of spotty operation, Tesla began offering a familiar fob last fall for an extra US $150. Yet that fob didn’t allow the “passive entry” of the smartphone system, requiring owners to push a button to enter or lock the car. 

Now it’s Lincoln’s turn. Ford’s luxury division will bring two SUVs to market this year, the midsize 2020 Aviator and compact 2020 Corsair. Traditionalists will still receive a standard fob, but adventurous types can pay extra for the Lincoln’s app-based, optional “Phone as a Key” system. 

The Ram 1500 Hybrid Merges Pickup Power With Sedan Fuel Efficiency

Post Syndicated from Lawrence Ulrich original https://spectrum.ieee.org/transportation/advanced-cars/2019s-top-10-tech-cars-ram-1500

Frugal with fuel in stop-and-go traffic

The 2019 Ram has been garnering early praise for several technologies unheard of in full-size pickup trucks: a Tesla-like touch screen, a coil-spring rear suspension and self-leveling air suspension. But its best tech trick is under the hood: mild hybrid power. It’s called eTorque, and it’s standard on every V-6 Ram and an option on Hemi V-8 models.

Mild hybrids can’t propel themselves on electricity alone, but they can supplement gasoline power and trim fuel consumption. On the Ram, a liquid-cooled motor/generator connects to the Pentastar V-6’s crankshaft to deliver an electric boost of 8.9 kilowatts (12 horsepower) and as many as 122 newton meters (90 pound-feet) of torque. It’s powered by a 48-volt electrical system, the new wave in automotive electricals, with a DC/DC converter and a compact, 0.4-kilowatt-hour lithium-ion battery.

That 48-V system permits the use of engine stop/start tech that cycles so seamlessly that it’s nearly undetectable: The Ram rolls from stoplights under electric power before it cranks the gasoline engine to whispery life, without the shuddering or noise that make typical stop/start systems so annoying.

Throw in an incredibly creamy ride, and a back seat (in Crew Cab models) with more legroom than any full-size luxury sedan, and you realize how far we’ve come from the days when the General Motors GMT 400 was hailed for having independent front suspension with torsion bars.

Ram says the eTorque system saves 5 centiliters (1.7 ounces) of fuel for every 90-second stop. Do that just 10 times a day and you’re conserving 190 liters (50 gallons) of fuel a year. It also saves energy through regenerative hybrid brakes. The latest, 227-kW (305-hp) Pentastar V-6 adds variable intake-valve lift and cam phasing that can run the efficient Atkinson combustion cycle, familiar from hybrids like the Toyota Prius. The 295-kW (395-hp) Hemi V-8 adds its own goodies, including fuel-saving cylinder deactivation, electronic mass dampers on frame rails and active cabin-noise cancellation, the latter two techs designed to erase telltale vibrations when the Ram runs on just four cylinders.

The upshot is the kind of fuel economy once associated with family cars. The V-6 Ram has an EPA fuel economy of 12.4 liters/100 kilometers (19 miles per gallon) on local roads and 9.8 L/100 km (24 mpg) on the highway, and an unmatched driving range of 1,000 km (624 miles) on a tank of gasoline. Even the burly V-8 eTorque model manages up to 17/23 mpg, in a truck that can tow a whopping 5,783 kilograms, or approximately one African bull elephant.

Ferrari’s Fastest-Ever Street-Legal V8 Takes Design and Tech Cues from Its F1 racers

Post Syndicated from Lawrence Ulrich original https://spectrum.ieee.org/transportation/advanced-cars/2019s-top-10-tech-cars-ferrari-488-pista

It’s a turbo with the satisfying sound of an air-breathing monster

When I get my first knee-wobbling glimpse of the 488 Pista and its zesty racing stripes, I’m not thinking about technology, I have to confess. But when I strap aboard the Pista at Ferrari’s fabled Fiorano test circuit in Maranello, Italy, I’m soon saying grazie for the sheer technical prowess of the fastest V-8 Ferrari ever produced.

A midmounted, dry-sump, twin-turbo V-8 spools up 529 kilowatts (710 horsepower) from just 3.9 liters of displacement, in a Ferrari that weighs just 1,382 kilograms (3,047 pounds). That’s 6 percent lighter than a 488 GTB, the standard version of Ferrari’s midengine marvel. The diet that slimmed down the Pista included carbon-fiber wheels that weigh 40 percent less than standard rims.

The result is a new idea of insanity in a street-legal Ferrari: The 0-to-100-kilometer-per-hour run (62 miles per hour) takes 2.85 seconds. You get to 200 km/h (124 mph) in 7.6 seconds, which is faster than many cars take to reach 100 km/h.

Engineers trimmed 23 kg (50 lbs) from the engine alone, using carbon-fiber intake plenums and titanium connecting rods, just like in Ferrari’s F1 racers. The engine’s total rotational inertia—created by its moving parts and by friction—is reduced by 17 percent for faster, more-joyous revving. The Inconel-alloy exhaust manifold is just 1 millimeter wide at its thinnest sections, and it saves nearly 9 kg (20 lbs). The design minimizes energy losses incurred when the engine pumps out exhaust. It also helps deliver the fortissimo sound that went missing in the GTB, a major challenge as supercars switch en masse to more-efficient turbocharged power plants.

Engineers also added more “color” to the sound of the engine by augmenting the richer, more-pleasing frequencies. Turbocharger speed sensors on each cylinder bank measure how well it’s working in real time to enable engine controllers to maximize power, regardless of altitude or ambient temperature.

The Ferrari takes aerodynamic and handling cues from Ferrari Challenge racers, along with the 488 GTBs that have dominated FIA Endurance Racing. Compared with a standard GTB, the Pista enjoys a huge 20 percent gain in aero efficiency, including up to 240 kg (529 lbs) of downforce at 200 km/h (124 mph). Giant carbon-ceramic brakes feel strong enough to halt Earth’s rotation. The S-Duct, a Ferrari showroom first, channels air through the front fascia and over the hood to clamp front tires to the road surface. Front radiators are inverted and canted rearward to direct hot air along the underbody but well away from side intercooler intakes.

As in Challenge cars, the engine is actually fed from the rear, where air intakes mounted just below the rear spoiler take advantage of the high-pressure atmosphere there; the 488’s signature cleavages in rear fenders are now put to use feeding air into turbo intercoolers and cooling the engine bay. The rear diffuser incorporates three active flaps that can rotate up to 14 degrees to minimize drag, hastening runs to the car’s top speed of 340 km/h (211 mph). The result is a track-day carnival.

The car’s coolest hand-me-down from racing tech may be the new “wall effect” rev limiter. Traditional engine-speed limiters, Ferrari says, cut off the fuel well before the engine gets to its redline. In the Pista, there’s no sudden slump in power, the dispiriting thrustus interruptus that you feel when a car’s engine bangs off the rev limiter. Instead, the Ferrari continues to accelerate right up to the engine’s peak, and holds it there. All 710 of these prancing ponies are on tap, anywhere from 6,750 rpm to the 8,000-rpm redline.

Ferrari will build just 500 Pistas for the world’s consumption. If only technology could make the Pista multiply while sharply reducing the price.

All-Electric Jaguar Earns its Racing Stripes

Post Syndicated from Lawrence Ulrich original https://spectrum.ieee.org/transportation/advanced-cars/2019s-top-10-tech-cars-jaguar-ipace-etrophy

A production EV, refitted for the racetrack

Making an electric car go fast in a straight line on the street is one thing; making one that can withstand the rigors of racing is another. Jaguar’s I-Pace eTrophy showed the difference in December by kicking off the world’s first all-electric, production-based championship with a historic run in Saudi Arabia. That eTrophy series, a global undercard for the FIA’s Formula E, included pro women—such as Katherine Legge with the Rahal Letterman Lanigan Racing team—competing against men in the kingdom for the first time.

I did my laps in that same car at the Silverstone circuit in the United Kingdom. And if the I-Pace isn’t the fastest race car I’ve tested—give this technology time, please—it was definitely the quietest. As I shot around Silverstone’s Stowe Circuit, I could barely hear the motor’s gentle whir above the groans of the suspension and the rumbling of the 56-centimeter (22-inch) Michelin Pilot Sport tires.

Jaguar’s Special Vehicle Operations has built 20 of these groundbreaking racers, which use as a starting point the same electric I-Pace you see in showrooms. The racers have the same 90-kilowatt-hour battery, for example, and also the same dual synchronous motors with 294 kilowatts (394 horsepower) and 700 newton meters of torque. Despite a 610-kilogram battery, the racers trim 225 kilos from the road-going version’s hefty 1,965 kg.

In this newfangled series, durability and safety matter as much as performance: Motors are designed to last for the series’ entire three years, or 30 races, with only the battery getting replaced after each season. The Terra 51 Charge Station, a DC fast charger with roughly 50 kW of power, is custom-designed to be compact and easily portable, for transport and racing. Those races last for 25 minutes plus one lap, or roughly 100 kilometers (62 miles), depending on the course.

I’m strapped into the car, sharing its safety cage with the battery, which is cradled inboard for safety. Gasoline fire has always been a prime danger in racing, but the Jaguar’s electrical system, with 389 volts and up to 550 amperes, brings its own challenges: If I crash, I’ll need to toggle separate switches that trigger two levels of electrical isolation and minimize the chance I’ll be electrocuted. Lights on the Jag’s exterior and dash glow green when there’s no live electricity but glow blue or red for potential threats. In that case, emergency workers are equipped with rubber gloves, mats, and specialized gear to aid the driver and car.

To maximize cooling, the race version has hood and fascia openings larger than those in the showroom car, as well as double the A/C system capacity, the better to cool the battery pack. Regenerative brakes create up to 0.4 g’s of deceleration. When the battery is fully charged, no energy recovery is possible, but as the race proceeds, drivers can adjust the regenerative brake over 11 settings. The results are pretty impressive for a two-metric-ton crossover: The racing I-Pace scoots to 97 km/h (60 mph) in 4.5 seconds, with a top speed twice as high, at 121 mph.

But this competition is also about putting Jaguar on the EV map, according to James Barclay, director for the Panasonic Jaguar Racing Team. “We have 9,500 engineers in [England] developing this technology,” Barclay says. “As a British company, we’re the first to have gone racing with our electric car, and we’re very proud of that.”

The New Lexus ES 350 Ditches Side View Mirrors for Digital Cameras

Post Syndicated from Lawrence Ulrich original https://spectrum.ieee.org/transportation/advanced-cars/2019s-top-10-tech-cars-lexus-es-350

Who needs mirrors when you’ve got cameras?

For years, automakers have teased us with sleek concept cars that ditch conventional side mirrors in favor of digital cameras. Now, Lexus becomes the first automaker to bring the technology to showrooms, albeit only in Japan for now: The all-new ES 350 sedan will beat the Audi E-tron’s “virtual mirrors” to market by mere months.

Lexus calls theirs the Digital Side-View Monitor: a pair of slender exterior stalks housing digital cameras that beam a high-definition view to 5.0-inch LCD screens mounted on either side of the cabin. The main benefit is to save fuel by reducing the aerodynamic drag of chunky conventional mirrors and damping the wind noise that’s been a longtime challenge for car designers. Lexus says the cameras and interior monitors deliver better visibility in foul weather, with the lenses heated and tucked into housings to ward off raindrops, ice, or snow. Drivers can adjust screen perspectives, or the camera can automatically adjust— zooming in when drivers hit their turn signals, or highlighting and alerting to cars that loom in blind spots.

Now, if only American motorists could get a peek. Though regulators in Japan and Europe have already approved the digital cameras, U.S. regulations prohibit them.

The Newest BMW 3 Takes the Worry Out of What’s in Your Rear View

Post Syndicated from Lawrence Ulrich original https://spectrum.ieee.org/transportation/advanced-cars/2019s-top-10-tech-cars-bmw-3series

Reverse assistant gets you right back where you started from

BMW’s franchise model, its 3-Series sport sedan, has been slipping in sales and reputation. It’s been hammered lately by the Alfa Romeo Giulia, among other sporty chariots. But an all-new 3-Series has armored itself for the battle with loads of new tech. First things first: The new 3-Series is fun to drive, in the way BMW fans demand. I learn this firsthand in Portugal while romping an M340i on the devilish Portimao circuit.

The car’s 285 kilowatts (382 horsepower) are a nearly 20 percent jump from last year, even though the car has a similar 3.0-liter in-line 6 at its core. The upgrades focus on the turbocharger: Fuel-injection pressure is nearly doubled, and the twin-scroll unit is lighter and more efficient, inhaling exhaust gas at a toasty 1,000 °C.

When drivers aren’t pushing the BMW’s limits, they can relax and enjoy the tech treats. Semiautonomous functions, including hands-off driving on highways, are managed through radar and three bundled cameras from MobilEye, an Israel-based Intel subsidiary that produces imagers and software for driver-assistance systems. Another feature, called Narrow Passage Support, keeps the BMW centered in white-knuckle maneuvers, such as driving between giant trucks. The built-in Intelligent Personal Assistant follows orders like an onboard Siri. Blue-laser-powered high beams can illuminate the road ahead for a remarkable 600 meters. And a plug-in hybrid version, designated 330e, heads to showrooms in 2020.

Now, let’s back up and talk about Reversing Assistant: At speeds below 35 kilometers per hour (22 miles per hour), the BMW records the car’s path in continuous 50-meter increments. Go ahead, drive along that dead-end forest path, a steep uphill driveway, the trickiest courtyard or parking garage. You can even park the BMW and get a good night’s sleep. When you return, the BMW will automatically steer itself to mirror the inbound path at up to 5.5 km/h, with the driver touching only the brake and throttle. The system worked like a charm, precisely reversing and scanning for pedestrians, cars, or obstacles, while I monitored its progress on a display screen; all without me having to crane my neck and worry about crunching into something. If every car had the BMW’s system, insurance claims due to reverse maneuvers gone wrong would drop faster than a shorn-off side mirror.

Like Magic, Land Rover Makes Its SUV’s Hood Disappear

Post Syndicated from Lawrence Ulrich original https://spectrum.ieee.org/cars-that-think/transportation/advanced-cars/like-magic-land-rover-makes-its-suvs-hood-disappear

The automaker’s new wheels-eye-view feature gets rid of the ultimate blind spot: the point where the rubber meets the road

Venturing off-road in a four-wheel-drive SUV can whiten the knuckles of even experienced drivers. So, even as engineers race to bring self-driving cars to paved roads, Land Rover is harnessing similar technology to help vehicles navigate the roughest terrain. The British automaker’s latest advance, which it calls Clear Sight Ground View, provided a literal eye-opener in Greece during my test of the sumptuous new Range Rover Evoque.

Like every new bit of semiautonomous tech, the groundbreaking system expands on an existing suite of hardware, sensors, and software. The latest Rovers could already robotically manage their acceleration, braking, suspensions, and four-wheel-drive traction well enough to tackle daunting climbs and descents and traverse terrain that would rattle the nerves of off-road novices. All that was left for the human driver was to steer, which reduced the mental and physical workload. Now, Clear Sight Ground View removes even more stress and uncertainty from the equation. At speeds up to 30 kilometers per hour (18 miles per hour), forward-facing cameras mounted on the side mirrors and front grille deliver a 180-degree view of the front wheels and the ground beneath and ahead of the vehicle—a perspective that’s typically obscured by the hood of a high-riding SUV.

This real-time view is fed to the Evoque’s center touch screen, along with a graphical, transparent overlay of the vehicle’s sides and corners. Computer controls slightly delay the images from the front camera and stitch them together with side-mirror views to create a seamless real-time feed. Rover engineers say the system makes the Evoque’s hood—or, these being Brits, its “bonnet”—virtually see-through. It’s definitely like having another set of eyes outside the vehicle. And that’s precisely the point.

Proponents of autonomous cars talk about replacing fallible drivers, but the Rover system could help replace another occupation: the spotter that traditionally guides an off-road adventurer through particularly tricky off-road obstacles. When I aimed the Evoque up a steep, narrow goat trail overlooking the Aegean Sea, the virtual view showed precisely where my front wheels were pointed, as well as the rocks and ruts I needed to clamber over or steer around. This eliminated any need to have a passenger climb out and hand-signal me as I picked my way up the slope. The system again came in handy on a vertigo-inducing crossing of a rusting, out-of-service railway bridge that spans the Corinth Canal, the spectacular shipping channel that separates the Greek mainland from the Peloponnese peninsula. The screen view offered precise, reassuring confirmation that my wheels were straddling the railroad tracks, and that I wasn’t at risk of making a roughly 90-meter plunge to the azure waters below.

Speaking of water, the all-new Evoque can ford bodies of water up to 60 centimeters (23.6 inches) deep—or 9.9 cm deeper than earlier models could withstand. To be on the safe side, a new ultrasonic sensor measures and displays the water’s depth as you proceed, again bypassing the need for guesswork, a pair of waders, or a sharp stick to poke into a rushing stream.

It’s all very clever. But for this off-roading fan, foolproofing the experience with electronic guardian angels raises some important questions: Isn’t doing it yourself half (or even more than half) the fun of trekking an SUV through challenging terrain? Isn’t experiencing the attendant sense of adventure and overcoming obstacles skillfully the point of the exercise? Would Lewis and Clark have preferred a mountain-lion sensor and a set of Google Maps? (OK, it’s a definite “yes” to the third question.)

Nathan Hoyt, U.S. communications chief at Jaguar Land Rover, the marque’s parent company, notes that engaging the various helpers is optional. Owners are welcome to do it the old-fashioned way, relying on their own skills and senses when traveling off-road. But expensive as Range Rovers are, Clear Sight Ground View will likely find its primary use in keeping them pristine when negotiating urban and suburban obstacles such as narrow passages or tall curbs. The viewpoint it provides lowers the odds of scraping a pricey alloy wheel on a curb or dinging a body panel on the base of a light post in a shopping mall parking lot.

Furthermore, says Hoyt, “the [new suite of driver assistance] systems will help you get up a steep driveway in winter, or across a muddy soccer pitch without getting stuck.” In other words, even if your posh Rover spends more time in valet lines than the rugged wilderness, these technologies may still come to the rescue.

10 visualizations to try in Amazon QuickSight with sample data

Post Syndicated from Karthik Kumar Odapally original https://aws.amazon.com/blogs/big-data/10-visualizations-to-try-in-amazon-quicksight-with-sample-data/

If you’re not already familiar with building visualizations for quick access to business insights using Amazon QuickSight, consider this your introduction. In this post, we’ll walk through some common scenarios with sample datasets to provide an overview of how you can connect yuor data, perform advanced analysis and access the results from any web browser or mobile device.

The following visualizations are built from the public datasets available in the links below. Before we jump into that, let’s take a look at the supported data sources, file formats and a typical QuickSight workflow to build any visualization.

Which data sources does Amazon QuickSight support?

At the time of publication, you can use the following data methods:

  • Connect to AWS data sources, including:
    • Amazon RDS
    • Amazon Aurora
    • Amazon Redshift
    • Amazon Athena
    • Amazon S3
  • Upload Excel spreadsheets or flat files (CSV, TSV, CLF, and ELF)
  • Connect to on-premises databases like Teradata, SQL Server, MySQL, and PostgreSQL
  • Import data from SaaS applications like Salesforce and Snowflake
  • Use big data processing engines like Spark and Presto

This list is constantly growing. For more information, see Supported Data Sources.

Answers in instants

SPICE is the Amazon QuickSight super-fast, parallel, in-memory calculation engine, designed specifically for ad hoc data visualization. SPICE stores your data in a system architected for high availability, where it is saved until you choose to delete it. Improve the performance of database datasets by importing the data into SPICE instead of using a direct database query. To calculate how much SPICE capacity your dataset needs, see Managing SPICE Capacity.

Typical Amazon QuickSight workflow

When you create an analysis, the typical workflow is as follows:

  1. Connect to a data source, and then create a new dataset or choose an existing dataset.
  2. (Optional) If you created a new dataset, prepare the data (for example, by changing field names or data types).
  3. Create a new analysis.
  4. Add a visual to the analysis by choosing the fields to visualize. Choose a specific visual type, or use AutoGraph and let Amazon QuickSight choose the most appropriate visual type, based on the number and data types of the fields that you select.
  5. (Optional) Modify the visual to meet your requirements (for example, by adding a filter or changing the visual type).
  6. (Optional) Add more visuals to the analysis.
  7. (Optional) Add scenes to the default story to provide a narrative about some aspect of the analysis data.
  8. (Optional) Publish the analysis as a dashboard to share insights with other users.

The following graphic illustrates a typical Amazon QuickSight workflow.

Visualizations created in Amazon QuickSight with sample datasets

Visualizations for a data analyst

Source:  https://data.worldbank.org/

Download and Resources:  https://datacatalog.worldbank.org/dataset/world-development-indicators

Data catalog:  The World Bank invests into multiple development projects at the national, regional, and global levels. It’s a great source of information for data analysts.

The following graph shows the percentage of the population that has access to electricity (rural and urban) during 2000 in Asia, Africa, the Middle East, and Latin America.

The following graph shows the share of healthcare costs that are paid out-of-pocket (private vs. public). Also, you can maneuver over the graph to get detailed statistics at a glance.

Visualizations for a trading analyst

Source:  Deutsche Börse Public Dataset (DBG PDS)

Download and resources:  https://aws.amazon.com/public-datasets/deutsche-boerse-pds/

Data catalog:  The DBG PDS project makes real-time data derived from Deutsche Börse’s trading market systems available to the public for free. This is the first time that such detailed financial market data has been shared freely and continually from the source provider.

The following graph shows the market trend of max trade volume for different EU banks. It builds on the data available on XETRA engines, which is made up of a variety of equities, funds, and derivative securities. This graph can be scrolled to visualize trade for a period of an hour or more.

The following graph shows the common stock beating the rest of the maximum trade volume over a period of time, grouped by security type.

Visualizations for a data scientist

Source:  https://catalog.data.gov/

Download and resources:  https://catalog.data.gov/dataset/road-weather-information-stations-788f8

Data catalog:  Data derived from different sensor stations placed on the city bridges and surface streets are a core information source. The road weather information station has a temperature sensor that measures the temperature of the street surface. It also has a sensor that measures the ambient air temperature at the station each second.

The following graph shows the present max air temperature in Seattle from different RWI station sensors.

The following graph shows the minimum temperature of the road surface at different times, which helps predicts road conditions at a particular time of the year.

Visualizations for a data engineer

Source:  https://www.kaggle.com/

Download and resources:  https://www.kaggle.com/datasnaek/youtube-new/data

Data catalog:  Kaggle has come up with a platform where people can donate open datasets. Data engineers and other community members can have open access to these datasets and can contribute to the open data movement. They have more than 350 datasets in total, with more than 200 as featured datasets. It has a few interesting datasets on the platform that are not present at other places, and it’s a platform to connect with other data enthusiasts.

The following graph shows the trending YouTube videos and presents the max likes for the top 20 channels. This is one of the most popular datasets for data engineers.

The following graph shows the YouTube daily statistics for the max views of video titles published during a specific time period.

Visualizations for a business user

Source:  New York Taxi Data

Download and resources:  https://data.cityofnewyork.us/Transportation/2016-Green-Taxi-Trip-Data/hvrh-b6nb

Data catalog: NYC Open data hosts some very popular open data sets for all New Yorkers. This platform allows you to get involved in dive deep into the data set to pull some useful visualizations. 2016 Green taxi trip dataset includes trip records from all trips completed in green taxis in NYC in 2016. Records include fields capturing pick-up and drop-off dates/times, pick-up and drop-off locations, trip distances, itemized fares, rate types, payment types, and driver-reported passenger counts.

The following graph presents maximum fare amount grouped by the passenger count during a period of time during a day. This can be further expanded to follow through different day of the month based on the business need.

The following graph shows the NewYork taxi data from January 2016, showing the dip in the number of taxis ridden on January 23, 2016 across all types of taxis.

A quick search for that date and location shows you the following news report:

Summary

Using Amazon QuickSight, you can see patterns across a time-series data by building visualizations, performing ad hoc analysis, and quickly generating insights. We hope you’ll give it a try today!

 


Additional Reading

If you found this post useful, be sure to check out Amazon QuickSight Adds Support for Combo Charts and Row-Level Security and Visualize AWS Cloudtrail Logs Using AWS Glue and Amazon QuickSight.


Karthik Odapally is a Sr. Solutions Architect in AWS. His passion is to build cost effective and highly scalable solutions on the cloud. In his spare time, he bakes cookies and cupcakes for family and friends here in the PNW. He loves vintage racing cars.

 

 

 

Pranabesh Mandal is a Solutions Architect in AWS. He has over a decade of IT experience. He is passionate about cloud technology and focuses on Analytics. In his spare time, he likes to hike and explore the beautiful nature and wild life of most divine national parks around the United States alongside his wife.

 

 

 

 

Ransomware Update: Viruses Targeting Business IT Servers

Post Syndicated from Roderick Bauer original https://www.backblaze.com/blog/ransomware-update-viruses-targeting-business-it-servers/

Ransomware warning message on computer

As ransomware attacks have grown in number in recent months, the tactics and attack vectors also have evolved. While the primary method of attack used to be to target individual computer users within organizations with phishing emails and infected attachments, we’re increasingly seeing attacks that target weaknesses in businesses’ IT infrastructure.

How Ransomware Attacks Typically Work

In our previous posts on ransomware, we described the common vehicles used by hackers to infect organizations with ransomware viruses. Most often, downloaders distribute trojan horses through malicious downloads and spam emails. The emails contain a variety of file attachments, which if opened, will download and run one of the many ransomware variants. Once a user’s computer is infected with a malicious downloader, it will retrieve additional malware, which frequently includes crypto-ransomware. After the files have been encrypted, a ransom payment is demanded of the victim in order to decrypt the files.

What’s Changed With the Latest Ransomware Attacks?

In 2016, a customized ransomware strain called SamSam began attacking the servers in primarily health care institutions. SamSam, unlike more conventional ransomware, is not delivered through downloads or phishing emails. Instead, the attackers behind SamSam use tools to identify unpatched servers running Red Hat’s JBoss enterprise products. Once the attackers have successfully gained entry into one of these servers by exploiting vulnerabilities in JBoss, they use other freely available tools and scripts to collect credentials and gather information on networked computers. Then they deploy their ransomware to encrypt files on these systems before demanding a ransom. Gaining entry to an organization through its IT center rather than its endpoints makes this approach scalable and especially unsettling.

SamSam’s methodology is to scour the Internet searching for accessible and vulnerable JBoss application servers, especially ones used by hospitals. It’s not unlike a burglar rattling doorknobs in a neighborhood to find unlocked homes. When SamSam finds an unlocked home (unpatched server), the software infiltrates the system. It is then free to spread across the company’s network by stealing passwords. As it transverses the network and systems, it encrypts files, preventing access until the victims pay the hackers a ransom, typically between $10,000 and $15,000. The low ransom amount has encouraged some victimized organizations to pay the ransom rather than incur the downtime required to wipe and reinitialize their IT systems.

The success of SamSam is due to its effectiveness rather than its sophistication. SamSam can enter and transverse a network without human intervention. Some organizations are learning too late that securing internet-facing services in their data center from attack is just as important as securing endpoints.

The typical steps in a SamSam ransomware attack are:

1
Attackers gain access to vulnerable server
Attackers exploit vulnerable software or weak/stolen credentials.
2
Attack spreads via remote access tools
Attackers harvest credentials, create SOCKS proxies to tunnel traffic, and abuse RDP to install SamSam on more computers in the network.
3
Ransomware payload deployed
Attackers run batch scripts to execute ransomware on compromised machines.
4
Ransomware demand delivered requiring payment to decrypt files
Demand amounts vary from victim to victim. Relatively low ransom amounts appear to be designed to encourage quick payment decisions.

What all the organizations successfully exploited by SamSam have in common is that they were running unpatched servers that made them vulnerable to SamSam. Some organizations had their endpoints and servers backed up, while others did not. Some of those without backups they could use to recover their systems chose to pay the ransom money.

Timeline of SamSam History and Exploits

Since its appearance in 2016, SamSam has been in the news with many successful incursions into healthcare, business, and government institutions.

March 2016
SamSam appears

SamSam campaign targets vulnerable JBoss servers
Attackers hone in on healthcare organizations specifically, as they’re more likely to have unpatched JBoss machines.

April 2016
SamSam finds new targets

SamSam begins targeting schools and government.
After initial success targeting healthcare, attackers branch out to other sectors.

April 2017
New tactics include RDP

Attackers shift to targeting organizations with exposed RDP connections, and maintain focus on healthcare.
An attack on Erie County Medical Center costs the hospital $10 million over three months of recovery.
Erie County Medical Center attacked by SamSam ransomware virus

January 2018
Municipalities attacked

• Attack on Municipality of Farmington, NM.
• Attack on Hancock Health.
Hancock Regional Hospital notice following SamSam attack
• Attack on Adams Memorial Hospital
• Attack on Allscripts (Electronic Health Records), which includes 180,000 physicians, 2,500 hospitals, and 7.2 million patients’ health records.

February 2018
Attack volume increases

• Attack on Davidson County, NC.
• Attack on Colorado Department of Transportation.
SamSam virus notification

March 2018
SamSam shuts down Atlanta

• Second attack on Colorado Department of Transportation.
• City of Atlanta suffers a devastating attack by SamSam.
The attack has far-reaching impacts — crippling the court system, keeping residents from paying their water bills, limiting vital communications like sewer infrastructure requests, and pushing the Atlanta Police Department to file paper reports.
Atlanta Ransomware outage alert
• SamSam campaign nets $325,000 in 4 weeks.
Infections spike as attackers launch new campaigns. Healthcare and government organizations are once again the primary targets.

How to Defend Against SamSam and Other Ransomware Attacks

The best way to respond to a ransomware attack is to avoid having one in the first place. If you are attacked, making sure your valuable data is backed up and unreachable by ransomware infection will ensure that your downtime and data loss will be minimal or none if you ever suffer an attack.

In our previous post, How to Recover From Ransomware, we listed the ten ways to protect your organization from ransomware.

  1. Use anti-virus and anti-malware software or other security policies to block known payloads from launching.
  2. Make frequent, comprehensive backups of all important files and isolate them from local and open networks. Cybersecurity professionals view data backup and recovery (74% in a recent survey) by far as the most effective solution to respond to a successful ransomware attack.
  3. Keep offline backups of data stored in locations inaccessible from any potentially infected computer, such as disconnected external storage drives or the cloud, which prevents them from being accessed by the ransomware.
  4. Install the latest security updates issued by software vendors of your OS and applications. Remember to patch early and patch often to close known vulnerabilities in operating systems, server software, browsers, and web plugins.
  5. Consider deploying security software to protect endpoints, email servers, and network systems from infection.
  6. Exercise cyber hygiene, such as using caution when opening email attachments and links.
  7. Segment your networks to keep critical computers isolated and to prevent the spread of malware in case of attack. Turn off unneeded network shares.
  8. Turn off admin rights for users who don’t require them. Give users the lowest system permissions they need to do their work.
  9. Restrict write permissions on file servers as much as possible.
  10. Educate yourself, your employees, and your family in best practices to keep malware out of your systems. Update everyone on the latest email phishing scams and human engineering aimed at turning victims into abettors.

Please Tell Us About Your Experiences with Ransomware

Have you endured a ransomware attack or have a strategy to avoid becoming a victim? Please tell us of your experiences in the comments.

The post Ransomware Update: Viruses Targeting Business IT Servers appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

The Challenges of Opening a Data Center — Part 1

Post Syndicated from Roderick Bauer original https://www.backblaze.com/blog/choosing-data-center/

Backblaze storage pod in new data center

This is part one of a series. The second part will be posted later this week. Use the Join button above to receive notification of future posts in this series.

Though most of us have never set foot inside of a data center, as citizens of a data-driven world we nonetheless depend on the services that data centers provide almost as much as we depend on a reliable water supply, the electrical grid, and the highway system. Every time we send a tweet, post to Facebook, check our bank balance or credit score, watch a YouTube video, or back up a computer to the cloud we are interacting with a data center.

In this series, The Challenges of Opening a Data Center, we’ll talk in general terms about the factors that an organization needs to consider when opening a data center and the challenges that must be met in the process. Many of the factors to consider will be similar for opening a private data center or seeking space in a public data center, but we’ll assume for the sake of this discussion that our needs are more modest than requiring a data center dedicated solely to our own use (i.e. we’re not Google, Facebook, or China Telecom).

Data center technology and management are changing rapidly, with new approaches to design and operation appearing every year. This means we won’t be able to cover everything happening in the world of data centers in our series, however, we hope our brief overview proves useful.

What is a Data Center?

A data center is the structure that houses a large group of networked computer servers typically used by businesses, governments, and organizations for the remote storage, processing, or distribution of large amounts of data.

While many organizations will have computing services in the same location as their offices that support their day-to-day operations, a data center is a structure dedicated to 24/7 large-scale data processing and handling.

Depending on how you define the term, there are anywhere from a half million data centers in the world to many millions. While it’s possible to say that an organization’s on-site servers and data storage can be called a data center, in this discussion we are using the term data center to refer to facilities that are expressly dedicated to housing computer systems and associated components, such as telecommunications and storage systems. The facility might be a private center, which is owned or leased by one tenant only, or a shared data center that offers what are called “colocation services,” and rents space, services, and equipment to multiple tenants in the center.

A large, modern data center operates around the clock, placing a priority on providing secure and uninterrrupted service, and generally includes redundant or backup power systems or supplies, redundant data communication connections, environmental controls, fire suppression systems, and numerous security devices. Such a center is an industrial-scale operation often using as much electricity as a small town.

Types of Data Centers

There are a number of ways to classify data centers according to how they will be used, whether they are owned or used by one or multiple organizations, whether and how they fit into a topology of other data centers; which technologies and management approaches they use for computing, storage, cooling, power, and operations; and increasingly visible these days: how green they are.

Data centers can be loosely classified into three types according to who owns them and who uses them.

Exclusive Data Centers are facilities wholly built, maintained, operated and managed by the business for the optimal operation of its IT equipment. Some of these centers are well-known companies such as Facebook, Google, or Microsoft, while others are less public-facing big telecoms, insurance companies, or other service providers.

Managed Hosting Providers are data centers managed by a third party on behalf of a business. The business does not own data center or space within it. Rather, the business rents IT equipment and infrastructure it needs instead of investing in the outright purchase of what it needs.

Colocation Data Centers are usually large facilities built to accommodate multiple businesses within the center. The business rents its own space within the data center and subsequently fills the space with its IT equipment, or possibly uses equipment provided by the data center operator.

Backblaze, for example, doesn’t own its own data centers but colocates in data centers owned by others. As Backblaze’s storage needs grow, Backblaze increases the space it uses within a given data center and/or expands to other data centers in the same or different geographic areas.

Availability is Key

When designing or selecting a data center, an organization needs to decide what level of availability is required for its services. The type of business or service it provides likely will dictate this. Any organization that provides real-time and/or critical data services will need the highest level of availability and redundancy, as well as the ability to rapidly failover (transfer operation to another center) when and if required. Some organizations require multiple data centers not just to handle the computer or storage capacity they use, but to provide alternate locations for operation if something should happen temporarily or permanently to one or more of their centers.

Organizations operating data centers that can’t afford any downtime at all will typically operate data centers that have a mirrored site that can take over if something happens to the first site, or they operate a second site in parallel to the first one. These data center topologies are called Active/Passive, and Active/Active, respectively. Should disaster or an outage occur, disaster mode would dictate immediately moving all of the primary data center’s processing to the second data center.

While some data center topologies are spread throughout a single country or continent, others extend around the world. Practically, data transmission speeds put a cap on centers that can be operated in parallel with the appearance of simultaneous operation. Linking two data centers located apart from each other — say no more than 60 miles to limit data latency issues — together with dark fiber (leased fiber optic cable) could enable both data centers to be operated as if they were in the same location, reducing staffing requirements yet providing immediate failover to the secondary data center if needed.

This redundancy of facilities and ensured availability is of paramount importance to those needing uninterrupted data center services.

Active/Passive Data Centers

Active/Active Data Centers

LEED Certification

Leadership in Energy and Environmental Design (LEED) is a rating system devised by the United States Green Building Council (USGBC) for the design, construction, and operation of green buildings. Facilities can achieve ratings of certified, silver, gold, or platinum based on criteria within six categories: sustainable sites, water efficiency, energy and atmosphere, materials and resources, indoor environmental quality, and innovation and design.

Green certification has become increasingly important in data center design and operation as data centers require great amounts of electricity and often cooling water to operate. Green technologies can reduce costs for data center operation, as well as make the arrival of data centers more amenable to environmentally-conscious communities.

The ACT, Inc. data center in Iowa City, Iowa was the first data center in the U.S. to receive LEED-Platinum certification, the highest level available.

ACT Data Center exterior

ACT Data Center exterior

ACT Data Center interior

ACT Data Center interior

Factors to Consider When Selecting a Data Center

There are numerous factors to consider when deciding to build or to occupy space in a data center. Aspects such as proximity to available power grids, telecommunications infrastructure, networking services, transportation lines, and emergency services can affect costs, risk, security and other factors that need to be taken into consideration.

The size of the data center will be dictated by the business requirements of the owner or tenant. A data center can occupy one room of a building, one or more floors, or an entire building. Most of the equipment is often in the form of servers mounted in 19 inch rack cabinets, which are usually placed in single rows forming corridors (so-called aisles) between them. This allows staff access to the front and rear of each cabinet. Servers differ greatly in size from 1U servers (i.e. one “U” or “RU” rack unit measuring 44.50 millimeters or 1.75 inches), to Backblaze’s Storage Pod design that fits a 4U chassis, to large freestanding storage silos that occupy many square feet of floor space.

Location

Location will be one of the biggest factors to consider when selecting a data center and encompasses many other factors that should be taken into account, such as geological risks, neighboring uses, and even local flight paths. Access to suitable available power at a suitable price point is often the most critical factor and the longest lead time item, followed by broadband service availability.

With more and more data centers available providing varied levels of service and cost, the choices increase each year. Data center brokers can be employed to find a data center, just as one might use a broker for home or other commercial real estate.

Websites listing available colocation space, such as upstack.io, or entire data centers for sale or lease, are widely used. A common practice is for a customer to publish its data center requirements, and the vendors compete to provide the most attractive bid in a reverse auction.

Business and Customer Proximity

The center’s closeness to a business or organization may or may not be a factor in the site selection. The organization might wish to be close enough to manage the center or supervise the on-site staff from a nearby business location. The location of customers might be a factor, especially if data transmission speeds and latency are important, or the business or customers have regulatory, political, tax, or other considerations that dictate areas suitable or not suitable for the storage and processing of data.

Climate

Local climate is a major factor in data center design because the climatic conditions dictate what cooling technologies should be deployed. In turn this impacts uptime and the costs associated with cooling, which can total as much as 50% or more of a center’s power costs. The topology and the cost of managing a data center in a warm, humid climate will vary greatly from managing one in a cool, dry climate. Nevertheless, data centers are located in both extremely cold regions and extremely hot ones, with innovative approaches used in both extremes to maintain desired temperatures within the center.

Geographic Stability and Extreme Weather Events

A major obvious factor in locating a data center is the stability of the actual site as regards weather, seismic activity, and the likelihood of weather events such as hurricanes, as well as fire or flooding.

Backblaze’s Sacramento data center describes its location as one of the most stable geographic locations in California, outside fault zones and floodplains.

Sacramento Data Center

Sometimes the location of the center comes first and the facility is hardened to withstand anticipated threats, such as Equinix’s NAP of the Americas data center in Miami, one of the largest single-building data centers on the planet (six stories and 750,000 square feet), which is built 32 feet above sea level and designed to withstand category 5 hurricane winds.

Equinix Data Center in Miami

Equinix “NAP of the Americas” Data Center in Miami

Most data centers don’t have the extreme protection or history of the Bahnhof data center, which is located inside the ultra-secure former nuclear bunker Pionen, in Stockholm, Sweden. It is buried 100 feet below ground inside the White Mountains and secured behind 15.7 in. thick metal doors. It prides itself on its self-described “Bond villain” ambiance.

Bahnhof Data Center under White Mountain in Stockholm

Usually, the data center owner or tenant will want to take into account the balance between cost and risk in the selection of a location. The Ideal quadrant below is obviously favored when making this compromise.

Cost vs Risk in selecting a data center

Cost = Construction/lease, power, bandwidth, cooling, labor, taxes
Risk = Environmental (seismic, weather, water, fire), political, economic

Risk mitigation also plays a strong role in pricing. The extent to which providers must implement special building techniques and operating technologies to protect the facility will affect price. When selecting a data center, organizations must make note of the data center’s certification level on the basis of regulatory requirements in the industry. These certifications can ensure that an organization is meeting necessary compliance requirements.

Power

Electrical power usually represents the largest cost in a data center. The cost a service provider pays for power will be affected by the source of the power, the regulatory environment, the facility size and the rate concessions, if any, offered by the utility. At higher level tiers, battery, generator, and redundant power grids are a required part of the picture.

Fault tolerance and power redundancy are absolutely necessary to maintain uninterrupted data center operation. Parallel redundancy is a safeguard to ensure that an uninterruptible power supply (UPS) system is in place to provide electrical power if necessary. The UPS system can be based on batteries, saved kinetic energy, or some type of generator using diesel or another fuel. The center will operate on the UPS system with another UPS system acting as a backup power generator. If a power outage occurs, the additional UPS system power generator is available.

Many data centers require the use of independent power grids, with service provided by different utility companies or services, to prevent against loss of electrical service no matter what the cause. Some data centers have intentionally located themselves near national borders so that they can obtain redundant power from not just separate grids, but from separate geopolitical sources.

Higher redundancy levels required by a company will of invariably lead to higher prices. If one requires high availability backed by a service-level agreement (SLA), one can expect to pay more than another company with less demanding redundancy requirements.

Stay Tuned for Part 2 of The Challenges of Opening a Data Center

That’s it for part 1 of this post. In subsequent posts, we’ll take a look at some other factors to consider when moving into a data center such as network bandwidth, cooling, and security. We’ll take a look at what is involved in moving into a new data center (including stories from Backblaze’s experiences). We’ll also investigate what it takes to keep a data center running, and some of the new technologies and trends affecting data center design and use. You can discover all posts on our blog tagged with “Data Center” by following the link https://www.backblaze.com/blog/tag/data-center/.

The second part of this series on The Challenges of Opening a Data Center will be posted later this week. Use the Join button above to receive notification of future posts in this series.

The post The Challenges of Opening a Data Center — Part 1 appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

[$] DIY biology

Post Syndicated from jake original https://lwn.net/Articles/747187/rss

A scientist with a rather unusual name, Meow-Ludo Meow-Meow, gave a talk at
linux.conf.au 2018
about the current trends in “do it yourself” (DIY) biology or
“biohacking”. He is perhaps most famous for being
prosecuted for implanting an Opal card RFID chip
into his hand; the
Opal card is used for public transportation fares in Sydney. He gave more
details about his implant as well as describing some other biohacking
projects in an engaging presentation.

Subway Elevators and Movie-Plot Threats

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2018/01/subway_elevator.html

Local residents are opposing adding an elevator to a subway station because terrorists might use it to detonate a bomb. No, really. There’s no actual threat analysis, only fear:

“The idea that people can then ride in on the subway with a bomb or whatever and come straight up in an elevator is awful to me,” said Claudia Ward, who lives in 15 Broad Street and was among a group of neighbors who denounced the plan at a recent meeting of the local community board. “It’s too easy for someone to slip through. And I just don’t want my family and my neighbors to be the collateral on that.”

[…]

Local residents plan to continue to fight, said Ms. Gerstman, noting that her building’s board decided against putting decorative planters at the building’s entrance over fears that shards could injure people in the event of a blast.

“Knowing that, and then seeing the proposal for giant glass structures in front of my building ­- ding ding ding! — what does a giant glass structure become in the event of an explosion?” she said.

In 2005, I coined the term “movie-plot threat” to denote a threat scenario that caused undue fear solely because of its specificity. Longtime readers of this blog will remember my annual Movie-Plot Threat Contests. I ended the contest in 2015 because I thought the meme had played itself out. Clearly there’s more work to be done.

Security Breaches Don’t Affect Stock Price

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2018/01/security_breach.html

Interesting research: “Long-term market implications of data breaches, not,” by Russell Lange and Eric W. Burger.

Abstract: This report assesses the impact disclosure of data breaches has on the total returns and volatility of the affected companies’ stock, with a focus on the results relative to the performance of the firms’ peer industries, as represented through selected indices rather than the market as a whole. Financial performance is considered over a range of dates from 3 days post-breach through 6 months post-breach, in order to provide a longer-term perspective on the impact of the breach announcement.

Key findings:

  • While the difference in stock price between the sampled breached companies and their peers was negative (1.13%) in the first 3 days following announcement of a breach, by the 14th day the return difference had rebounded to + 0.05%, and on average remained positive through the period assessed.
  • For the differences in the breached companies’ betas and the beta of their peer sets, the differences in the means of 8 months pre-breach versus post-breach was not meaningful at 90, 180, and 360 day post-breach periods.

  • For the differences in the breached companies’ beta correlations against the peer indices pre- and post-breach, the difference in the means of the rolling 60 day correlation 8 months pre- breach versus post-breach was not meaningful at 90, 180, and 360 day post-breach periods.

  • In regression analysis, use of the number of accessed records, date, data sensitivity, and malicious versus accidental leak as variables failed to yield an R2 greater than 16.15% for response variables of 3, 14, 60, and 90 day return differential, excess beta differential, and rolling beta correlation differential, indicating that the financial impact on breached companies was highly idiosyncratic.

  • Based on returns, the most impacted industries at the 3 day post-breach date were U.S. Financial Services, Transportation, and Global Telecom. At the 90 day post-breach date, the three most impacted industries were U.S. Financial Services, U.S. Healthcare, and Global Telecom.

The market isn’t going to fix this. If we want better security, we need to regulate the market.

Note: The article is behind a paywall. An older version is here. A similar article is here.

AWS IoT, Greengrass, and Machine Learning for Connected Vehicles at CES

Post Syndicated from Jeff Barr original https://aws.amazon.com/blogs/aws/aws-iot-greengrass-and-machine-learning-for-connected-vehicles-at-ces/

Last week I attended a talk given by Bryan Mistele, president of Seattle-based INRIX. Bryan’s talk provided a glimpse into the future of transportation, centering around four principle attributes, often abbreviated as ACES:

Autonomous – Cars and trucks are gaining the ability to scan and to make sense of their environments and to navigate without human input.

Connected – Vehicles of all types have the ability to take advantage of bidirectional connections (either full-time or intermittent) to other cars and to cloud-based resources. They can upload road and performance data, communicate with each other to run in packs, and take advantage of traffic and weather data.

Electric – Continued development of battery and motor technology, will make electrics vehicles more convenient, cost-effective, and environmentally friendly.

Shared – Ride-sharing services will change usage from an ownership model to an as-a-service model (sound familiar?).

Individually and in combination, these emerging attributes mean that the cars and trucks we will see and use in the decade to come will be markedly different than those of the past.

On the Road with AWS
AWS customers are already using our AWS IoT, edge computing, Amazon Machine Learning, and Alexa products to bring this future to life – vehicle manufacturers, their tier 1 suppliers, and AutoTech startups all use AWS for their ACES initiatives. AWS Greengrass is playing an important role here, attracting design wins and helping our customers to add processing power and machine learning inferencing at the edge.

AWS customer Aptiv (formerly Delphi) talked about their Automated Mobility on Demand (AMoD) smart vehicle architecture in a AWS re:Invent session. Aptiv’s AMoD platform will use Greengrass and microservices to drive the onboard user experience, along with edge processing, monitoring, and control. Here’s an overview:

Another customer, Denso of Japan (one of the world’s largest suppliers of auto components and software) is using Greengrass and AWS IoT to support their vision of Mobility as a Service (MaaS). Here’s a video:

AWS at CES
The AWS team will be out in force at CES in Las Vegas and would love to talk to you. They’ll be running demos that show how AWS can help to bring innovation and personalization to connected and autonomous vehicles.

Personalized In-Vehicle Experience – This demo shows how AWS AI and Machine Learning can be used to create a highly personalized and branded in-vehicle experience. It makes use of Amazon Lex, Polly, and Amazon Rekognition, but the design is flexible and can be used with other services as well. The demo encompasses driver registration, login and startup (including facial recognition), voice assistance for contextual guidance, personalized e-commerce, and vehicle control. Here’s the architecture for the voice assistance:

Connected Vehicle Solution – This demo shows how a connected vehicle can combine local and cloud intelligence, using edge computing and machine learning at the edge. It handles intermittent connections and uses AWS DeepLens to train a model that responds to distracted drivers. Here’s the overall architecture, as described in our Connected Vehicle Solution:

Digital Content Delivery – This demo will show how a customer uses a web-based 3D configurator to build and personalize their vehicle. It will also show high resolution (4K) 3D image and an optional immersive AR/VR experience, both designed for use within a dealership.

Autonomous Driving – This demo will showcase the AWS services that can be used to build autonomous vehicles. There’s a 1/16th scale model vehicle powered and driven by Greengrass and an overview of a new AWS Autonomous Toolkit. As part of the demo, attendees drive the car, training a model via Amazon SageMaker for subsequent on-board inferencing, powered by Greengrass ML Inferencing.

To speak to one of my colleagues or to set up a time to see the demos, check out the Visit AWS at CES 2018 page.

Some Resources
If you are interested in this topic and want to learn more, the AWS for Automotive page is a great starting point, with discussions on connected vehicles & mobility, autonomous vehicle development, and digital customer engagement.

When you are ready to start building a connected vehicle, the AWS Connected Vehicle Solution contains a reference architecture that combines local computing, sophisticated event rules, and cloud-based data processing and storage. You can use this solution to accelerate your own connected vehicle projects.

Jeff;

Tracking People Without GPS

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2017/12/tracking_people_5.html

Interesting research:

The trick in accurately tracking a person with this method is finding out what kind of activity they’re performing. Whether they’re walking, driving a car, or riding in a train or airplane, it’s pretty easy to figure out when you know what you’re looking for.

The sensors can determine how fast a person is traveling and what kind of movements they make. Moving at a slow pace in one direction indicates walking. Going a little bit quicker but turning at 90-degree angles means driving. Faster yet, we’re in train or airplane territory. Those are easy to figure out based on speed and air pressure.

After the app determines what you’re doing, it uses the information it collects from the sensors. The accelerometer relays your speed, the magnetometer tells your relation to true north, and the barometer offers up the air pressure around you and compares it to publicly available information. It checks in with The Weather Channel to compare air pressure data from the barometer to determine how far above sea level you are. Google Maps and data offered by the US Geological Survey Maps provide incredibly detailed elevation readings.

Once it has gathered all of this information and determined the mode of transportation you’re currently taking, it can then begin to narrow down where you are. For flights, four algorithms begin to estimate the target’s location and narrows down the possibilities until its error rate hits zero.

If you’re driving, it can be even easier. The app knows the time zone you’re in based on the information your phone has provided to it. It then accesses information from your barometer and magnetometer and compares it to information from publicly available maps and weather reports. After that, it keeps track of the turns you make. With each turn, the possible locations whittle down until it pinpoints exactly where you are.

To demonstrate how accurate it is, researchers did a test run in Philadelphia. It only took 12 turns before the app knew exactly where the car was.

This is a good example of how powerful synthesizing information from disparate data sources can be. We spend too much time worried about individual data collection systems, and not enough about analysis techniques of those systems.

Research paper.

What is HAMR and How Does It Enable the High-Capacity Needs of the Future?

Post Syndicated from Andy Klein original https://www.backblaze.com/blog/hamr-hard-drives/

HAMR drive illustration

During Q4, Backblaze deployed 100 petabytes worth of Seagate hard drives to our data centers. The newly deployed Seagate 10 and 12 TB drives are doing well and will help us meet our near term storage needs, but we know we’re going to need more drives — with higher capacities. That’s why the success of new hard drive technologies like Heat-Assisted Magnetic Recording (HAMR) from Seagate are very relevant to us here at Backblaze and to the storage industry in general. In today’s guest post we are pleased to have Mark Re, CTO at Seagate, give us an insider’s look behind the hard drive curtain to tell us how Seagate engineers are developing the HAMR technology and making it market ready starting in late 2018.

What is HAMR and How Does It Enable the High-Capacity Needs of the Future?

Guest Blog Post by Mark Re, Seagate Senior Vice President and Chief Technology Officer

Earlier this year Seagate announced plans to make the first hard drives using Heat-Assisted Magnetic Recording, or HAMR, available by the end of 2018 in pilot volumes. Even as today’s market has embraced 10TB+ drives, the need for 20TB+ drives remains imperative in the relative near term. HAMR is the Seagate research team’s next major advance in hard drive technology.

HAMR is a technology that over time will enable a big increase in the amount of data that can be stored on a disk. A small laser is attached to a recording head, designed to heat a tiny spot on the disk where the data will be written. This allows a smaller bit cell to be written as either a 0 or a 1. The smaller bit cell size enables more bits to be crammed into a given surface area — increasing the areal density of data, and increasing drive capacity.

It sounds almost simple, but the science and engineering expertise required, the research, experimentation, lab development and product development to perfect this technology has been enormous. Below is an overview of the HAMR technology and you can dig into the details in our technical brief that provides a point-by-point rundown describing several key advances enabling the HAMR design.

As much time and resources as have been committed to developing HAMR, the need for its increased data density is indisputable. Demand for data storage keeps increasing. Businesses’ ability to manage and leverage more capacity is a competitive necessity, and IT spending on capacity continues to increase.

History of Increasing Storage Capacity

For the last 50 years areal density in the hard disk drive has been growing faster than Moore’s law, which is a very good thing. After all, customers from data centers and cloud service providers to creative professionals and game enthusiasts rarely go shopping looking for a hard drive just like the one they bought two years ago. The demands of increasing data on storage capacities inevitably increase, thus the technology constantly evolves.

According to the Advanced Storage Technology Consortium, HAMR will be the next significant storage technology innovation to increase the amount of storage in the area available to store data, also called the disk’s “areal density.” We believe this boost in areal density will help fuel hard drive product development and growth through the next decade.

Why do we Need to Develop Higher-Capacity Hard Drives? Can’t Current Technologies do the Job?

Why is HAMR’s increased data density so important?

Data has become critical to all aspects of human life, changing how we’re educated and entertained. It affects and informs the ways we experience each other and interact with businesses and the wider world. IDC research shows the datasphere — all the data generated by the world’s businesses and billions of consumer endpoints — will continue to double in size every two years. IDC forecasts that by 2025 the global datasphere will grow to 163 zettabytes (that is a trillion gigabytes). That’s ten times the 16.1 ZB of data generated in 2016. IDC cites five key trends intensifying the role of data in changing our world: embedded systems and the Internet of Things (IoT), instantly available mobile and real-time data, cognitive artificial intelligence (AI) systems, increased security data requirements, and critically, the evolution of data from playing a business background to playing a life-critical role.

Consumers use the cloud to manage everything from family photos and videos to data about their health and exercise routines. Real-time data created by connected devices — everything from Fitbit, Alexa and smart phones to home security systems, solar systems and autonomous cars — are fueling the emerging Data Age. On top of the obvious business and consumer data growth, our critical infrastructure like power grids, water systems, hospitals, road infrastructure and public transportation all demand and add to the growth of real-time data. Data is now a vital element in the smooth operation of all aspects of daily life.

All of this entails a significant infrastructure cost behind the scenes with the insatiable, global appetite for data storage. While a variety of storage technologies will continue to advance in data density (Seagate announced the first 60TB 3.5-inch SSD unit for example), high-capacity hard drives serve as the primary foundational core of our interconnected, cloud and IoT-based dependence on data.

HAMR Hard Drive Technology

Seagate has been working on heat assisted magnetic recording (HAMR) in one form or another since the late 1990s. During this time we’ve made many breakthroughs in making reliable near field transducers, special high capacity HAMR media, and figuring out a way to put a laser on each and every head that is no larger than a grain of salt.

The development of HAMR has required Seagate to consider and overcome a myriad of scientific and technical challenges including new kinds of magnetic media, nano-plasmonic device design and fabrication, laser integration, high-temperature head-disk interactions, and thermal regulation.

A typical hard drive inside any computer or server contains one or more rigid disks coated with a magnetically sensitive film consisting of tiny magnetic grains. Data is recorded when a magnetic write-head flies just above the spinning disk; the write head rapidly flips the magnetization of one magnetic region of grains so that its magnetic pole points up or down, to encode a 1 or a 0 in binary code.

Increasing the amount of data you can store on a disk requires cramming magnetic regions closer together, which means the grains need to be smaller so they won’t interfere with each other.

Heat Assisted Magnetic Recording (HAMR) is the next step to enable us to increase the density of grains — or bit density. Current projections are that HAMR can achieve 5 Tbpsi (Terabits per square inch) on conventional HAMR media, and in the future will be able to achieve 10 Tbpsi or higher with bit patterned media (in which discrete dots are predefined on the media in regular, efficient, very dense patterns). These technologies will enable hard drives with capacities higher than 100 TB before 2030.

The major problem with packing bits so closely together is that if you do that on conventional magnetic media, the bits (and the data they represent) become thermally unstable, and may flip. So, to make the grains maintain their stability — their ability to store bits over a long period of time — we need to develop a recording media that has higher coercivity. That means it’s magnetically more stable during storage, but it is more difficult to change the magnetic characteristics of the media when writing (harder to flip a grain from a 0 to a 1 or vice versa).

That’s why HAMR’s first key hardware advance required developing a new recording media that keeps bits stable — using high anisotropy (or “hard”) magnetic materials such as iron-platinum alloy (FePt), which resist magnetic change at normal temperatures. Over years of HAMR development, Seagate researchers have tested and proven out a variety of FePt granular media films, with varying alloy composition and chemical ordering.

In fact the new media is so “hard” that conventional recording heads won’t be able to flip the bits, or write new data, under normal temperatures. If you add heat to the tiny spot on which you want to write data, you can make the media’s coercive field lower than the magnetic field provided by the recording head — in other words, enable the write head to flip that bit.

So, a challenge with HAMR has been to replace conventional perpendicular magnetic recording (PMR), in which the write head operates at room temperature, with a write technology that heats the thin film recording medium on the disk platter to temperatures above 400 °C. The basic principle is to heat a tiny region of several magnetic grains for a very short time (~1 nanoseconds) to a temperature high enough to make the media’s coercive field lower than the write head’s magnetic field. Immediately after the heat pulse, the region quickly cools down and the bit’s magnetic orientation is frozen in place.

Applying this dynamic nano-heating is where HAMR’s famous “laser” comes in. A plasmonic near-field transducer (NFT) has been integrated into the recording head, to heat the media and enable magnetic change at a specific point. Plasmonic NFTs are used to focus and confine light energy to regions smaller than the wavelength of light. This enables us to heat an extremely small region, measured in nanometers, on the disk media to reduce its magnetic coercivity,

Moving HAMR Forward

HAMR write head

As always in advanced engineering, the devil — or many devils — is in the details. As noted earlier, our technical brief provides a point-by-point short illustrated summary of HAMR’s key changes.

Although hard work remains, we believe this technology is nearly ready for commercialization. Seagate has the best engineers in the world working towards a goal of a 20 Terabyte drive by 2019. We hope we’ve given you a glimpse into the amount of engineering that goes into a hard drive. Keeping up with the world’s insatiable appetite to create, capture, store, secure, manage, analyze, rapidly access and share data is a challenge we work on every day.

With thousands of HAMR drives already being made in our manufacturing facilities, our internal and external supply chain is solidly in place, and volume manufacturing tools are online. This year we began shipping initial units for customer tests, and production units will ship to key customers by the end of 2018. Prepare for breakthrough capacities.

The post What is HAMR and How Does It Enable the High-Capacity Needs of the Future? appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Remote Hack of a Boeing 757

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2017/12/remote_hack_of_.html

Last month, the DHS announced that it was able to remotely hack a Boeing 757:

“We got the airplane on Sept. 19, 2016. Two days later, I was successful in accomplishing a remote, non-cooperative, penetration,” said Robert Hickey, aviation program manager within the Cyber Security Division of the DHS Science and Technology (S&T) Directorate.

“[Which] means I didn’t have anybody touching the airplane, I didn’t have an insider threat. I stood off using typical stuff that could get through security and we were able to establish a presence on the systems of the aircraft.” Hickey said the details of the hack and the work his team are doing are classified, but said they accessed the aircraft’s systems through radio frequency communications, adding that, based on the RF configuration of most aircraft, “you can come to grips pretty quickly where we went” on the aircraft.

Amazon QuickSight Update – Geospatial Visualization, Private VPC Access, and More

Post Syndicated from Jeff Barr original https://aws.amazon.com/blogs/aws/amazon-quicksight-update-geospatial-visualization-private-vpc-access-and-more/

We don’t often recognize or celebrate anniversaries at AWS. With nearly 100 services on our list, we’d be eating cake and drinking champagne several times a week. While that might sound like fun, we’d rather spend our working hours listening to customers and innovating. With that said, Amazon QuickSight has now been generally available for a little over a year and I would like to give you a quick update!

QuickSight in Action
Today, tens of thousands of customers (from startups to enterprises, in industries as varied as transportation, legal, mining, and healthcare) are using QuickSight to analyze and report on their business data.

Here are a couple of examples:

Gemini provides legal evidence procurement for California attorneys who represent injured workers. They have gone from creating custom reports and running one-off queries to creating and sharing dynamic QuickSight dashboards with drill-downs and filtering. QuickSight is used to track sales pipeline, measure order throughput, and to locate bottlenecks in the order processing pipeline.

Jivochat provides a real-time messaging platform to connect visitors to website owners. QuickSight lets them create and share interactive dashboards while also providing access to the underlying datasets. This has allowed them to move beyond the sharing of static spreadsheets, ensuring that everyone is looking at the same and is empowered to make timely decisions based on current data.

Transfix is a tech-powered freight marketplace that matches loads and increases visibility into logistics for Fortune 500 shippers in retail, food and beverage, manufacturing, and other industries. QuickSight has made analytics accessible to both BI engineers and non-technical business users. They scrutinize key business and operational metrics including shipping routes, carrier efficient, and process automation.

Looking Back / Looking Ahead
The feedback on QuickSight has been incredibly helpful. Customers tell us that their employees are using QuickSight to connect to their data, perform analytics, and make high-velocity, data-driven decisions, all without setting up or running their own BI infrastructure. We love all of the feedback that we get, and use it to drive our roadmap, leading to the introduction of over 40 new features in just a year. Here’s a summary:

Looking forward, we are watching an interesting trend develop within our customer base. As these customers take a close look at how they analyze and report on data, they are realizing that a serverless approach offers some tangible benefits. They use Amazon Simple Storage Service (S3) as a data lake and query it using a combination of QuickSight and Amazon Athena, giving them agility and flexibility without static infrastructure. They also make great use of QuickSight’s dashboards feature, monitoring business results and operational metrics, then sharing their insights with hundreds of users. You can read Building a Serverless Analytics Solution for Cleaner Cities and review Serverless Big Data Analytics using Amazon Athena and Amazon QuickSight if you are interested in this approach.

New Features and Enhancements
We’re still doing our best to listen and to learn, and to make sure that QuickSight continues to meet your needs. I’m happy to announce that we are making seven big additions today:

Geospatial Visualization – You can now create geospatial visuals on geographical data sets.

Private VPC Access – You can now sign up to access a preview of a new feature that allows you to securely connect to data within VPCs or on-premises, without the need for public endpoints.

Flat Table Support – In addition to pivot tables, you can now use flat tables for tabular reporting. To learn more, read about Using Tabular Reports.

Calculated SPICE Fields – You can now perform run-time calculations on SPICE data as part of your analysis. Read Adding a Calculated Field to an Analysis for more information.

Wide Table Support – You can now use tables with up to 1000 columns.

Other Buckets – You can summarize the long tail of high-cardinality data into buckets, as described in Working with Visual Types in Amazon QuickSight.

HIPAA Compliance – You can now run HIPAA-compliant workloads on QuickSight.

Geospatial Visualization
Everyone seems to want this feature! You can now take data that contains a geographic identifier (country, city, state, or zip code) and create beautiful visualizations with just a few clicks. QuickSight will geocode the identifier that you supply, and can also accept lat/long map coordinates. You can use this feature to visualize sales by state, map stores to shipping destinations, and so forth. Here’s a sample visualization:

To learn more about this feature, read Using Geospatial Charts (Maps), and Adding Geospatial Data.

Private VPC Access Preview
If you have data in AWS (perhaps in Amazon Redshift, Amazon Relational Database Service (RDS), or on EC2) or on-premises in Teradata or SQL Server on servers without public connectivity, this feature is for you. Private VPC Access for QuickSight uses an Elastic Network Interface (ENI) for secure, private communication with data sources in a VPC. It also allows you to use AWS Direct Connect to create a secure, private link with your on-premises resources. Here’s what it looks like:

If you are ready to join the preview, you can sign up today.

Jeff;