All posts by Jeremy Hsu

Risk Dashboard Could Help the Power Grid Manage Renewables

Post Syndicated from Jeremy Hsu original

To fully embrace wind and solar power unless, grid operators need to be able to predict and manage the variability that comes from changes in the wind or clouds dimming sunlight.

One solution may come from a $2-million project backed by the U.S. Department of Energy that aims to develop a risk dashboard for handling more complex power grid scenarios.


Grid operators now use dashboards that report the current status of the power grid and show the impacts of large disturbances—such as storms and other weather contingencies—along with regional constraints in flow and generation. The new dashboard being developed by Columbia University researchers and funded by the Advanced Research Projects Agency–Energy (ARPA-E) would improve upon existing dashboards by modeling more complex factors. This could help the grid better incorporate both renewable power sources and demand response programs that encourage consumers to use less electricity during peak periods.

“[Y]ou have to operate the grid in a way that is looking forward in time and that accepts that there will be variability—you have to start talking about what people in finance would call risk,” says Daniel Bienstock, professor of industrial engineering and operations research, and professor of applied physics and applied mathematics at Columbia University.

The new dashboard would not necessarily help grid operators prepare for catastrophic black swan events that might happen only once in 100 years. Instead, Bienstock and his colleagues hope to apply some lessons from financial modeling to measure and manage risk associated with more common events that could strain the capabilities of the U.S. regional power grids managed by independent system operators (ISOs). The team plans to build and test an alpha version of the dashboard within two years, before demonstrating the dashboard for ISOs and electric utilities in the third year of the project.

Variability already poses a challenge to modern power grids that were designed to handle steady power output from conventional power plants to meet an anticipated level of demand from consumers. Power grids usually rely on gas turbine generators to kick in during peak periods of power usage or to provide backup to intermittent wind and solar power.

But such generators may not provide a fast enough response to compensate for the expected variability in power grids that include more renewable power sources and demand response programs driven by fickle human behavior. In the worst cases, grid operators may shut down power to consumers and create deliberate blackouts in order to protect the grid’s physical equipment.

One of the dashboard project’s main goals involves developing mathematical and statistical models that can quantify the risk from having greater uncertainty in the power grid. Such models would aim to simulate different scenarios based on conditions—such as changes in weather or power demand—that could stress the power grid. Repeatedly playing out such scenarios would force grid operators to fine-tune and adapt their operational plans to handle such surprises in real life.

For example, one scenario might involve a solar farm generating 10 percent less power and a wind farm generating 30 percent more power within a short amount of time, Bienstock explains. The combination of those factors might mean too much power begins flowing on a particular power line and the line subsequently starts running hot at the risk of damage.

Such models would only be as good as the data that trains them. Some ISOs and electric utilities have already been gathering useful data from the power grid for years. Those that already have more experience dealing with the variability of renewable power have been the most proactive. But many of the ISOs are reluctant to share such data with outsiders.

“One of the ISOs has told us that they will let us run our code on their data provided that we actually physically go to their office, but they will not give us the data to play with,” Bienstock says. 

For this project, ARPA-E has been working with one ISO to produce synthetic data covering many different scenarios based on historical data. The team is also using publicly available data on factors such as solar irradiation, cloud cover, wind strength, and the power generation capabilities of solar panels and wind turbines.

“You can look at historical events and then you can design stress that’s somehow compatible with what we observe in the past,” says Agostino Capponi, associate professor of industrial engineering and operations research at Columbia University and external consultant for the U.S. Commodity Futures Trading Commission.

A second big part of the dashboard project involves developing tools that grid operators could use to help manage the risks that come from dealing with greater uncertainty. Capponi is leading the team’s effort to design customized energy volatility contracts that could allow grid operators to buy such contracts for a fixed amount and receive compensation for all the variance that occurs over a historical period of time.

But he acknowledged that financial contracts designed to help offset risk in the stock market won’t apply in a straightforward manner to the realities of the power grid that include delays in power transmission, physical constraints, and weather events.

“You cannot really directly use existing financial contracts because in finance you don’t have to take into account the physics of the power grid,” Capponi says.

Once the new dashboard is up and running, it could begin to help grid operators deal with both near-term and long-term challenges for the U.S. power grid. One recent example comes from the current COVID-19 pandemic and associated human behavioral changes—such as more people working from home—having already increased variability in energy consumption across New York City and other parts of the United States. In the future, the risk dashboard might help grid operators quickly identify areas at higher risk of suffering from imbalances between supply and demand and act quickly to avoid straining the grid or having blackouts.

Knowing the long-term risks in specific regions might also drive more investment in additional energy storage technologies and improved transmission lines to help offset such risks. The situation is different for every grid operator’s particular region, but the researchers hope that their dashboard can eventually help level the speed bumps as the U.S. power grid moves toward using more renewable power.

“The ISOs have different levels of renewable penetration, and so they have different exposures and visibility to risk,” Bienstock says. “But this is just the right time to be doing this sort of thing.”

Survey Finds Americans Skeptical of Contact Tracing Apps

Post Syndicated from Jeremy Hsu original

Confusion and skepticism may confound efforts to make use of digital contact tracing technologies during the COVID-19 pandemic. A recent survey found that just 42 percent of American respondents support using so-called contact tracing apps—an indication of a lack of confidence that could weaken or even derail effective deployment of such technologies.

How the Pandemic Impacts U.S. Electricity Usage

Post Syndicated from Jeremy Hsu original

As the COVID-19 outbreak swept through Manhattan and the surrounding New York City boroughs earlier this year, electricity usage dropped as businesses shuttered and people hunkered down in their homes. Those changes in human behavior became visible from space as the nighttime lights of the city that never sleeps dimmed by 40 percent between February and April.

That striking visualization of the COVID-19 impact on U.S. electricity consumption came from NASA’s “Black Marble” satellite data. U.S. and Chinese researchers are currently using such data sources in what they describe as an unprecedented effort to study how electricity consumption across the United States has been changing in response to the pandemic. One early finding suggests that mobility in the retail sector—defined as daily visits to retail establishments—is an especially significant factor in the reduction of electricity consumption seen across all major U.S. regional markets.

“I was previously not aware that there is such a strong correlation between the mobility in the retail sector and the public health data on the electricity consumption,” says Le Xie, professor in electrical and computer engineering and assistant director of energy digitization at the Texas A&M Energy Institute. “So that is a key finding.”

Xie and his colleagues from Texas A&M, MIT, and Tsinghua University in Beijing, China, are publicly sharing their Coronavirus Disease-Electricity Market Data Aggregation (COVID-EMDA) project and the software codes they have used in their analyses in an online Github repository. They first uploaded a preprint paper describing their initial analyses to arXiv on 11 May 2020. 

Most previous studies that focused on public health and electricity consumption tried to examine whether changes in electricity usage could provide an early warning sign of health issues. But when the U.S. and Chinese researchers first put their heads together on studying COVID-19 impacts, they did not find other prior studies that had examined how a pandemic can affect electricity consumption.

Beyond using the NASA satellite imagery of the nighttime lights, the COVID-EMDA project also taps additional sources of data about the major U.S. electricity markets from regional transmission organizations, weather patterns, COVID-19 cases, and the anonymized GPS locations of cellphone users.

“Before when people study electricity, they look at data on the electricity domain, perhaps the weather, maybe the economy, but you would have never thought about things like your cell phone data or mobility data or the public health data from COVID cases,” Xie says. “These are traditionally totally unrelated data sets, but in these very special circumstances they all suddenly became very relevant.”

The unique compilation of different data sources has already helped the researchers spot some interesting patterns. The most notable finding suggests that the largest portion of the drop in electricity consumption likely comes from the drop in people’s daily visits to retail establishments as individuals begin early adoption of practicing social distancing and home isolation. By comparison, the number of new confirmed COVID-19 cases does not seem to have a strong direct influence on changes in electricity consumption.

The Northeastern region of the U.S. electricity sector that includes New York City seems to be experiencing the most volatile changes so far during the pandemic. Xie and his colleagues hypothesize that larger cities with higher population density and commercial activity would likely see bigger COVID-19 impacts on their electricity consumption. But they plan to continue monitoring electricity consumption changes in all the major regions as new COVID-19 hotspots have emerged outside the New York City area.

The biggest limitation of such an analysis comes from the lack of available higher-resolution data on electricity consumption. Each of the major regional transmission organizations publishes power load and price numbers daily for their electricity markets, but this reflects a fairly large geographic area that often covers multiple states. 

“For example, if we could know exactly how much electricity is used in each of the commercial, industrial, and residential categories in a city, we could have a much clearer picture of what is going on,” Xie says.

That could change in the near future. Some Texas utility companies have already approached the COVID-EMDA group about possibly sharing such higher-resolution data on electricity consumption for future analyses. The researchers have also heard from economists curious about analyzing and perhaps predicting near-term economic activities based on electricity consumption changes during the pandemic.

One of the next big steps is to “develop a predictive model with high confidence to estimate the impact to electricity consumption due to social-distancing policies,” Xie says. “This could potentially help the public policy people and [regional transmission organizations] to prepare for similar situations in the future.”

Spherical Solar Cells Soak Up Scattered Sunlight

Post Syndicated from Jeremy Hsu original

Flat solar panels still face big limitations when it comes to making the most of the available sunlight each day. A new spherical solar cell design aims to boost solar power harvesting potential from nearly every angle without requiring expensive moving parts to keep tracking the sun’s apparent movement across the sky. 

Optical Atomic Clocks Are Ready to Redefine Time

Post Syndicated from Jeremy Hsu original

Optical atomic clocks will likely redefine the international standard for measuring a second in time. They are far more accurate and stable than the current standard, which is based on microwave atomic clocks.

Now, researchers in the United States have figured out how to convert high-performance signals from optical clocks into a microwave signal that can more easily find practical use in modern electronic systems.

How to Protect Privacy While Mining Millions of Patient Records

Post Syndicated from Jeremy Hsu original

When people in the United Kingdom began dying from COVID-19, researchers saw an urgent need to understand all the possible factors contributing to such deaths. So in six weeks, a team of software developers, clinicians, and academics created an open-source platform designed to securely analyze millions of electronic health records while protecting patient privacy. 

Preventing AI From Divulging Its Own Secrets

Post Syndicated from Jeremy Hsu original

One of the sneakiest ways to spill the secrets of a computer system involves studying its pattern of power usage while it performs operations. That’s why researchers have begun developing ways to shield the power signatures of AI systems from prying eyes.

Sabrewing Cargo Drone Rises to Air Force Challenge

Post Syndicated from Jeremy Hsu original

Named after a dragon from the “Game of Thrones” fantasy series, the Rhaegal-A won’t be making its mark by burninating the countryside. Instead the hybrid-electric cargo drone capable of taking off and landing like a helicopter is in the spotlight today during a U.S. Air Force conference about “flying car” technologies. 

Coronavirus Pandemic Prompts Privacy-Conscious Europe to Collect Phone Data

Post Syndicated from Jeremy Hsu original

Amid the coronavirus pandemic, even privacy-conscious European governments have asked telecom companies for residents’ phone location data in hopes of understanding whether national social distancing measures such as stay-at-home orders and business closures are having any effect on the spread of COVID-19.

Some of the hardest-hit countries, including Italy and Spain, are now open to proposals for mobile apps that can make contact tracing more efficient and alert people who have come into contact with someone infected by the novel coronavirus.

How to Detect a Government’s Hand Behind Internet Shutdowns

Post Syndicated from Jeremy Hsu original

Internet shutdowns that affect entire regions or countries and cost billions of dollars annually have become a widespread phenomenon, especially as various governments wield them like a blunt club to restrict citizens’ access to online information.

Some governments deploy Internet shutdowns in an attempt to suppress protests, while Iraq’s Ministry of Education even orders shutdowns to prevent cheating during national school exams. The trick for independent observers trying to keep track of it all involves figuring out the difference between government-ordered shutdowns versus other causes of Internet outages.

In early 2020, the five-person team behind the nongovernmental organization NetBlocks was watching dips in Internet connectivity happening in a particular region of China over several months. That could have sparked suspicion that China’s online censors—who restrict access to certain online content as part of China’s “Great Firewall”—were perhaps throttling some popular online services or social media networks. But the NetBlocks team’s analysis showed that such patterns likely had to do with businesses shutting down or limiting operations to comply with government efforts aimed at containing the coronavirus outbreak that has since become a pandemic.

“When you’re investigating an internet shutdown, you need to work from both ends to conclusively verify that incident has happened, and to understand why it’s happened,” says Alp Toker, executive director of NetBlocks. “This means ruling out different types of outages.”

NetBlocks is among the independent research groups trying to keep an eye on the growing prevalence of Internet shutdowns. Since it formed in 2016, the London-based NetBlocks has expanded its focus from Turkey and the Middle East to other parts of the world by using remote measurement techniques. These include analytics software that monitors how well millions of phones and other devices can access certain online websites and services, along with both hardware probes plugged into local routers and an Internet browser probe that anyone can use to check their local connectivity.

But NetBlocks also relies upon what Toker describes as a more hands-on investigation to manually check out various incidents. That could mean checking in with local engineers or Internet service providers who are in a position to help confirm or rule out certain lines of inquiry. This combined approach has helped NetBlocks investigate all sorts of causes of Internet shutdowns, including major hurricanes, nationwide power outages in Venezuela and cuts in undersea Internet cables affecting Africa and the Middle East. Each of these types of outages provides data that NetBlocks is using to train machine learning algorithms in hopes of better automating detection and analysis of different events.

“Each of the groups that’s currently monitoring Internet censorship uses a different technical approach and can observe different aspects of what’s happening,” says Zachary Weinberg, a postdoctoral researcher at the University of Massachusetts Amherst and member of the Information Controls Lab (ICLab) project. “We’re working with them on combining all of our data sets to get a more complete picture.

ICLab relies heavily on a network of commercial virtual private networks (VPNs) to gain observation points that provide a window into Internet connectivity in each country, along with a handful of human volunteers based around the world. These VPN observation points can do bandwidth-intensive tests and collect lots of data on network traffic without endangering volunteers in certain countries. But one limitation of this approach is that VPN locations in commercial data centers are sometimes not subject to the same Internet censorship affecting residential networks and mobile networks.

If a check turns up possible evidence of a network shutdown, ICLab’s internal monitoring alerts the team. The researchers use manual confirmation checks to make sure it’s a government-ordered shutdown action and not something like a VPN service malfunction. “We have some ad-hoc rules in our code to try to distinguish these possibilities, and plans to dig into the data [collected] so far and come up with something more principled,” Weinberg says.

The Open Observatory of Network Interference (OONI) takes a more decentralized, human-reliant approach to measuring Internet censorship and outages. OONI’s six-person team has developed and refined a computer software tool called OONI probe that people can download and run to can check local Internet connectivity with a number of websites, including a global test list of internationally relevant websites (such as Facebook) and a country-specific test list. 

The OONI project began when members of the Tor Project, the nonprofit organization that oversees the Tor network designed to enable people to use the Internet anonymously, began creating “ad hoc scripts” to investigate blocking of Tor software and other examples of Internet censorship, says Arturo Filasto, lead developer of OONI. Since 2012, that has evolved into the free and open-source OONI probe with an openly-documented methodology explaining how it measures Internet censorship, along with a frequently updated database that anyone can search.

“We eventually consolidated [that] into the software that now tens of thousands of people run all over the world to collect their own evidence of Internet censorship and contribute to this growing pool of open data that anybody can use to research and investigate various forms of information controls on the Internet,” Filasto says.

Beyond the tens of thousands of active monthly users, hundreds of millions of people have downloaded the OONI probe. That probe is currently available as a mobile app and for desktop Linux and macOS users who don’t mind using the command-line interface, but the team aims to launch a more user-friendly desktop program for Windows and macOS users in April 2020. 

Other groups have their own approaches. The CensoredPlanet lab at the University of Michigan uses echo servers that exist primarily to bounce messages back to senders as observation points. The Cooperative Association for Internet Data Analysis (CAIDA) at the University of California in San Diego monitors global online traffic involving the Border Gateway Protocol, which backbone routers use to communicate with each other. 

On the low-tech side, news articles and word-of-mouth reports from ordinary people can also provide valuable internet outage data for websites such as the Internet Shutdown Tracker run by the Software Freedom Law Centre in New Delhi, India. But the Internet Shutdown Tracker website also invites mobile users to download and install the OONI probe tool as a way of helping gather more data on regional and city-level Internet shutdowns ordered by India’s government.

Whatever their approach, most of the groups tracking Internet shutdowns and online censorship still consist of small teams with budget constraints. For example, ICLab’s team would like to speed up and automate much of their process, but their budget is reliant in large part upon getting grants from the U.S. National Science Foundation. They also have limited data storage that restricts them to checking each country about two or three times a week on average to collect detailed cycles of measurements—amounting to about 500 megabytes of raw data per country. 

Another challenge comes on the data collection side. People may face personal risk in downloading and using OONI probe or similar tools in some countries, especially if the government’s laws regard such actions as illegal or even akin to espionage. This is why the OONI team openly warns about the risk up front as part of what they consider their informed consent process, and even require mobile users to complete a quiz before starting to use the OONI probe app.

“Thanks to the fact that many people are running OONI probe in China and Iran, we’ve been able to uncover a lot of really interesting and important cases of Internet censorship that we wouldn’t otherwise have known,” Filasto says. “So we are very grateful to the brave users of OONI probe that have gathered these important measurements.”

Recent trends in both government information control strategies and the broader Internet landscape may also complicate the work of such groups. Governments in countries such as China, Russia, and Iran have begun moving away from network-level censorship toward embedding censorship policies within large social media platforms and chat systems such as Tencent’s WeChat in China. Detecting more subtle censorship within these platforms represents an even bigger challenge than collecting evidence of a region-wide Internet shutdown.

“We have to create accounts on all these systems, which in some cases requires proof of physical-space identity, and then we have to automate access to them, which the platforms intentionally make as difficult as possible,” Weinberg says. “And then we have to figure out whether someone’s post isn’t showing up because of censorship, or because the ‘algorithm’ decided our test account wouldn’t be interested in it.”

In 2019, large-scale Internet shutdowns affecting entire countries occurred alongside the shift toward “more nuanced Internet disruptions that happen on different layers,” Toker says. The NetBlocks team is refining its analytical capability to home in on different types of outages by learning more about the daily pattern of Internet traffic that reflects each country’s normal economic activities. But Toker is also hoping that his group and others can continue forging international cooperation to study these issues together. For now, NetBlocks relies upon community contributions, the technical community, and volunteers.

“There are bubbles of expertise in different parts of the world, and those haven’t necessarily combined, so from where we’ve been coming I think those bridges are just starting to be built,” Toker says. “And that means really getting engineers together from different fields and different backgrounds, whether it’s electrical engineering or Internet engineering.”

New Cryptography Method Promising Perfect Secrecy Is Met With Skepticism

Post Syndicated from Jeremy Hsu original

In the ongoing race to make and break digital codes, the idea of perfect secrecy has long hovered on the horizon like a mirage. A recent research paper has attracted both interest and skepticism for describing how to achieve perfect secrecy in communications by using specially-patterned silicon chips to generate one-time keys that are impossible to recreate.

How India, the World’s Largest Democracy, Shuts Down the Internet

Post Syndicated from Jeremy Hsu original

When government officials in India decided to shut down the Internet, software engineers working for an IT and data analytics firm lost half a day of work and fell behind in delivering a project for clients based in London. A hotel was unable to pay its employees or manage online bookings for tourists. A major hospital delayed staff salary payments and restricted its medical services to the outpatient and emergency departments.  

AI System Warns Pedestrians Wearing Headphones About Passing Cars

Post Syndicated from Jeremy Hsu original

Journal Watch report logo, link to report landing page

How can headphone-wearing pedestrians tune out the chaotic world around them without compromising their own safety? One solution may come from the pedestrian equivalent of a vehicle collision warning system that aims to detect nearby vehicles based purely on sound.

Fear of Internet Censorship Hangs Over Hong Kong Protests

Post Syndicated from Jeremy Hsu original

Fears about online censorship have grown since Hong Kong government officials raised the possibility of curbing Internet freedom to suppress a city-wide protest movement that has led to increasingly violent clashes between riot police and some protesters.

Microsoft’s AI Research Draws Controversy Over Possible Disinformation Use

Post Syndicated from Jeremy Hsu original

AI capable of automatically posting relevant comments on news articles has raised concerns that the technology could empower online disinformation campaigns designed to influence public opinion and national elections. The AI research in question, conducted by Microsoft Research Asia and Beihang University in China, became the subject of controversy even prior to the paper’s scheduled presentation at a major AI conference this week.

Google’s Quantum Tech Milestone Excites Scientists and Spurs Rivals

Post Syndicated from Jeremy Hsu original

Quantum computing can already seem like the realm of big business these days, with tech giants such as Google, IBM, and Intel developing quantum tech hardware. But even as rivals reacted to Google’s announcement of having shown quantum computing’s advantage over the most powerful supercomputer, scientists have welcomed the demonstration as providing crucial experimental evidence to back up theoretical research in quantum physics.

What Google’s Quantum Supremacy Claim Means for Quantum Computing

Post Syndicated from Jeremy Hsu original

Google’s claim to have demonstrated quantum supremacy—one of the earliest and most hotly anticipated milestones on the long road toward practical quantum computing—was supposed to make its official debut in a prestigious science journal. Instead, an early leak of the research paper has sparked a frenzy of media coverage and some misinformed speculation about when quantum computers will be ready to crack the world’s computer security algorithms.

How Language Shapes Password Security

Post Syndicated from Jeremy Hsu original

No matter the differences in language and culture, both Chinese- and English-language Internet users apparently find common ground in using easily guessable password variants of “123456.” But a recent study comparing password patterns among the two languages also found notable and unique features in Chinese passwords that have big implications for Internet security beyond China.