As the old saying goes, good things come in pairs, 好事成双！ The month of May marks a double celebration in China for our customers, partners and Cloudflare.
First and Foremost
A Beijing Customer Appreciation Cocktail was held in the heart of Beijing at Yintai Centre Xiu Rooftop Garden Bar on the 10 May 2019, an RSVP event graced by our supportive group of partners and customers.
We have been blessed with almost 10 years of strong growth at Cloudflare – sharing our belief in providing access to internet security and performance to customers of all sizes and industries. This success has been the result of collaboration between our developers, our product team as represented today by our special guest, Jen Taylor, our Global Head of Product, Business Leaders Xavier Cai, Head of China business, and Aliza Knox Head of our APAC Business, James Ball our Head of Solutions Engineers for APAC, most importantly, by the trust and faith that our partners, such as Baidu, and customers have placed in us.
Double Happiness, 双喜
On the same week, we embarked on another exciting journey in China with our grand office opening at WeWork. Beijing team consists of functions from Customer Development to Solutions Engineering and Customer Success lead by Xavier, Head of China business. The team has grown rapidly in size by double since it started last year.
We continue to invest in China and to grow our customer base, and importantly our methods for supporting our customers, here are well. Those of us who came from different parts of the world, are also looking to learn from the wisdom and experience of our customers in this market. And to that end, we look forward to many more years of openness, trust, and mutual success.
During February, China’s National Copyright Administration (NCAC) announced that it would be upping efforts to deal with copyright infringement.
On top of a promise to “dig deep” into the sources of piracy and “sternly investigate” online platforms that help to distribute pirated content, the NCAC said it would also target unauthorized “camming”.
Camming, the act of recording movies in theaters with video cameras, has been a major headache for the entertainment industries for decades. Illegal copies often hit the Internet within hours of a movie’s premiere, as was the case last month with Avengers: Endgame.
While the NCAC clearly couldn’t do anything about that serious event, the question remains whether physical deterrents (such as bag searches and action against complicit theater owners) can also be augmented by technical measures.
Before Endgame dramatically hit the web, the China-based partnership of Ogilvy and Focus Film Media, part of Focus Media Group, announced that they had developed a new system to prevent camming taking in place in cinemas.
“Originality is the soul of the film industry and the foundation from which it thrives upon; it is our job to protect this originality,” said Jason Jiang, Founder and Chairman of Focus Media Group.
“We are delighted to have gone beyond a conventional approach and develop the ‘Piracy Blockr,’ which allows us to address the problem in a discrete but effective way, ensuring that the film industry is protected for years to come.”
The image above, although clearly mocked up, provides an idea of how the system is supposed to work. A watermark, invisible to the viewer, is captured by camcorders when an attempt is made to record the screen.
So how does it work? TorrentFreak spoke with Ogilvy to find out.
“There is a lot more to light than what mere human eyes can detect, but a device in your pocket can help you see beyond your biological limits. Our eyes can only detect colors of light that we see as a rainbow, primarily shades of red, orange, yellow, green, blue, indigo, and violet,” says Silvia Zhang, Ogilvy Marketing & Communications Manager.
“So while our naked eyes can’t pick up on the wavelength of infrared light, the sensors in your phones and cameras can – essentially making the invisible visible.”
Anyone with a smartphone can easily see what the system is about. Simply press a button on an infrared remote control and point it at the camera lens and the image on the screen will display the infrared light emitted by the device. The camera can ‘see’ the infrared light, we can’t.
“We used this to our advantage to combat the multi-billion dollar illegal cam recording industry by embedding panels of infrared light powered watermarks, which we call the ‘Piracy Blockr’, behind cinema screens in China,” Zhang adds.
The idea of using infrared light to foil pirates isn’t new. A report dating back almost 10 years reveals that Japan’s National Institute of Informatics had teamed up with Sharp to pulse infrared light through cinema screens to disturb digital recording devices.
Since we haven’t heard of any such devices actually being deployed in cinemas, we asked Ogilvy how many screens its system currently ‘protects’ in China. The company didn’t respond to our question, despite repeated attempts.
We also asked how the Piracy Blockr system is able to defeat determined cammers who attach infrared filters to their devices. The company didn’t respond to that question either. A request for a real-life image or video clip of Piracy Blockr in action received the same response.
Some research appears to have been carried out in India (pdf) which considered the challenges presented by pirates who deploy infrared filtering but the problem clearly isn’t straightforward. If it was, someone would be making millions by now while resigning ‘camming’ to history.
As for Piracy Blockr, we won’t be holding our breath while waiting for a live demo.
In 2016, a hacker group calling itself the ShadowBrokers released a trove of 2013 NSA hacking tools and related documents. Most people believe it is a front for the Russian government. Since, then the vulnerabilities and tools have been used by both government and criminals, and put the NSA’s ability to secure its own cyberweapons seriously into question.
Now we havelearned that the Chinese used the tools fourteen months before the Shadow Brokers released them.
Does this mean that both the Chinese and the Russians stole the same set of NSA tools? Did the Russians steal them from the Chinese, who stole them from us? Did it work the other way? I don’t think anyone has any idea. But this certainly illustrates how dangerous it is for the NSA — or US Cyber Command — to hoard zero-day vulnerabilities.
Supply chain security is an insurmountably hard problem. The recent focus is on Chinese 5G equipment, but the problem is much broader. This opinion piece looks at undersea communications cables:
But now the Chinese conglomerate Huawei Technologies, the leading firm working to deliver 5G telephony networks globally, has gone to sea. Under its Huawei Marine Networks component, it is constructing or improving nearly 100 submarine cables around the world. Last year it completed a cable stretching nearly 4,000 miles from Brazil to Cameroon. (The cable is partly owned by China Unicom, a state-controlled telecom operator.) Rivals claim that Chinese firms are able to lowball the bidding because they receive subsidies from Beijing.
Just as the experts are justifiably concerned about the inclusion of espionage “back doors” in Huawei’s 5G technology, Western intelligence professionals oppose the company’s engagement in the undersea version, which provides a much bigger bang for the buck because so much data rides on so few cables.
This shouldn’t surprise anyone. For years, the US and the Five Eyes have had a monopoly on spying on the Internet around the globe. Other countries want in.
As I have repeatedly said, we need to decide if we are going to build our future Internet systems for security or surveillance. Either everyone gets to spy, or no one gets to spy. And I believe we must choose security over surveillance, and implement a defense-dominant strategy.
“Real knowledge is to know the extent of one’s ignorance.” ― Confucius
Don’t tell our CEO, Matthew Prince, but the first day I interviewed at Cloudflare I had a $9.00 phone in my pocket, a knock-off similar to a Nokia 5140, but the UI was all in Chinese characters—that phone was a fitting symbol for my technical prowess. At that time in my career I could send emails and use Google, but that was about the extent of my tech skill set. The only code I’d ever seen was in the Matrix, Apple computers confused me, and I was working as a philosophy lecturer at The University of California, Santa Cruz. So, you know, I was pretty much the ideal candidate for a deeply technical, Silicon Valley startup.
This was in 2013. I had just returned from two years of Peace Corps service in the far Southwest of China approaching the Himalayan plateau. That experience gave me the confidence to walk into Cloudflare’s office knowing that I would be good for the job despite the gaps in my knowledge. My early training in philosophy plus my Peace Corps service gave me a blueprint for learning and figuring things out when thrown into the deep end (it turns out that I love being thrown into the deep end and learning to swim).
I had no idea that this first meeting with Matthew would eventually lead me back to China, this time riding on the cloud of a fast-growing Silicon Valley tech giant.
Two years earlier, eighty Peace Corps Volunteers and myself landed in the capital of Sichuan province, Chengdu. The vast majority of us, myself included, spoke zero Mandarin and only knew about China from books and a few news snippets here and there. The Chinese staff members that greeted us at the Peace Corps China headquarters on the Sichuan University campus affectionately called us “baby pandas”, because we were cute and fairly incompetent in terms of operating in China.
Our mission was to help China meet its need for trained men and women—specifically to teach college level students English and train qualified Teachers of English as a foreign language instructors (TEFL instructors). We were also there to promote a better understanding of Americans abroad, and to do our best to gain some understanding of China and its people.
Thus began two years of deep learning and profound personal growth.
When I think about the most important aspects of my time in China, there are three fundamentals that I come back to:
The importance of learning the language and culture
The importance of 关系 (guanxi) or personal connections and relationships
The necessity of being resourceful
The most successful Peace Corps volunteers in my cohort were the ones that learned to speak Mandarin well, understood enough about Chinese culture to operate effectively in their schools and communities, had built important personal and professional relationships, and had figured out how to survive in Southwest China and be useful as English language resources and American cultural liaisons. There was a steep learning curve.
Peace Corps Service in China has four phases more or less. Phase one, Pre Service Training (PST), took place at Sichuan University. We were all living with Chinese host families, taking 8-9 hours of Mandarin class each day, learning about Chinese culture, and being trained as TEFL instructors. It is an intense period of learning against a backdrop of tremendous culture shock, jet lag, and general confusion of how to be an American in Southwest China.
After three months of well taught crash courses, I was sent out to the college where I would spend the next two years of my service. That first night, after I unpacked my bags and took a shower, the reality of my life decisions came crashing down. This was going to be *very* hard. I was alone with millions and millions of Chinese people in remote Sichuan. Phase two was about to begin.
Getting familiar with the college where I was to spend two years was another steep learning curve. I was introduced to the colleagues I’d be teaching with as well as the school administrators, and, most importantly, I was introduced to my students. I got lucky, the English department at my school was small, and I only had 20-30 students in each of my classes. I met with them 4 times a week for two hours a day, so I had ample time to really get to know them and work with them one-on-one in the classroom, during office hours, and over spicy Sichuan dinners.
That first year of service I studied Mandarin as if my life depended on it—because it sort of did. Few people, i.e. my students and colleagues, spoke English in rural Sichuan. As I was able to communicate better in Mandarin, my understanding of the culture grew and so did my relationships with folks at my school and community.
In an effort to understand more about the culture I was living in, I gave myself an education in Chinese philosophy starting with Confucius (孔子) and the Daoist like Laozi (老子) and Zhuangzi (庄子), and I also looked into Buddhism. Since the world’s wisdom traditions contain universal principles that transcend time and culture, these readings gave me subtle insights into the Chinese way of life. I learned that Confucianism is the invisible glue holding much of Chinese society together. And while Confucius spoke to Chinese society and how people ought to act, his contemporary, Laozi, considered the founder of Daoism, spoke to the Chinese soul via the Dao de Jing (道德经).
Apropos of philosophy, one beautiful Chinese proverb I found in my reading goes: “Only those who take leisurely what the rest of the world is concerned about, can be concerned about with the rest of the world takes leisurely”. A calligraphy artist at my school gifted me a piece of work expressing this:
I also learned early on in my service what my students needed: authentic opportunities to express themselves in English, understanding and encouragement, and a solid English text book that employed the latest pedagogical techniques for learning a foreign language. Since my Mandarin was slow going, my students had all sorts of authentic opportunities to speak to me in English. They ended up helping me translate a lot that first year as I navigated my life on campus. As for encouragement, I would often talk to them in my developing and broken Mandarin in front of the class. I messed up words and tones constantly, and they laughed (hard) and then kindly corrected me. In this way, I showed them that learning is all about making mistakes, and that it is fine to get it wrong as you begin. There is no other way to learn a language (or anything else). The last part, providing a solid textbook, would be more tricky.
I received enough training during PST to have some good ideas for teaching English as a foreign language, but I had no experience writing a language textbook. What I ended up doing was replicating the structure of the textbooks I was using to learn Mandarin: a dialogue which incorporates a few new vocab words, a list of those new vocab words, grammar practice using grammatical structures from the dialogue, and then photos of relevant objects or scenes that would allow students to use new vocabulary words to describe the photos using new words and structures. I would record these dialogues and then distribue the audio file to my students so they could hear my pronunciation.
We’d work with this dialogue, vocabulary, and grammar all week, then on Fridays I’d put them in a “language line”. Sort of like speed dating, but they would have to hold a conversation with their classmates around the topic of that week and use the new vocab words. I’d listen in and help guide them. Then at the end of class, we’d form a line and I’d ask each one of them a question individually that they had to answer before they could leave the classroom. This pushed each student into learning so that they could actually speak English confidently to a native English speaker. It was a rewarding project.
My students were super smart and diligent, and week after week their English level was going up. I was able to hold natural conversations with them while speaking slow, and my Mandarin was progressing to the point that I could clarify things in Mandarin to aid their English learning. And so I learned how to teach English.
I consider all of the second year of service phase three. It is in that second year that volunteers can do really great work. My language level was high enough to really communicate with my community and explore China more, I had a basic structure for teaching and kept honing it to fit the needs of my students, and I developed a lot of really important relationships with the administrators at my school and other wonderful folks in the area.
Phase four is the return to the US. Something that no one told me about Peace Corps service before I joined is that you actually sign up for three years, not two. And that the third year, the first year back home after service, would be the most challenging by far … readjusting to life in the US, starting up or continuing a career, feeling a million miles behind peers who cranked through two extra years in a work world. All of this while trying to work on one of the most important goals of the Peace Corps—Goal 3—helping Americans better understand China through my experiences. I’m doing this every chance I get. This blog is a part of fulfilling Goal 3.
My service in China impacted me in profound ways. I have a love and respect for China that was born of close contact with the wonderful people, culture, philosophy, and language I was steeped in. And it gave me a clear experience of my ability to grow and change and acquire new skills swiftly. By the end of my time, I could confidently hold a conversation in Mandarin, I could read sections of Chinese newspapers, I had written an English text book for my students, and I made so many friends. All of that came from slow, diligent, hard work—and finding the necessary resources to get things done for my students in non-obvious ways. I had a clearly outlined experience of what diligence and time can do, and I knew deep in me is the potential to learn, adapt, and grow into almost anything.
Two years of remote Peace Corps work (which, despite being among millions of Chinese people, is often an isolating experience) gave me ample time to reflect on my life. While I find teaching deeply rewarding and I love the study of philosophy, I felt that I needed a different pool to swim in than academia. I thought that the private sector would likely offer the most opportunity, so when I came back to the US, I decided to move to San Francisco and aim for a job in tech. I figured that would be like plunging into the ocean, and I was keen to see where the global economic currents might take me.
In the first few weeks I was back in the US I set up 4-5 informational interviews each week. I spoke to people at Google and Square, folks working in event planning, in finance, in HR, in construction, etc. Then one of my colleagues at the university mentioned that their friend (Matthew) had a tech startup called “Cloudflare” and could maybe use some help writing stuff. I followed up right away.
Career Change: From teaching to tech – How Hard Can It Be?
Despite hours of Googling “What is a Cloudflare?”, I was utterly and completely out of my depth when Matthew explained to me what the company does. Before my interview with him, I had done my homework memorizing definitions for acronyms like CDN, DNS, DDoS, and API, but I didn’t really know what they were. The instructions I received before the interview were to learn a bit about how Cloudflare works, and “Don’t wear a suit and tie”. This was a time in Cloudflare history when we had about 60 employees, about 30 data centers, and a bit of duct tape in the office pressing extension cords into the carpet.
I was intimidated speaking to Matthew the first time. He is an amazingly accomplished and incredibly intelligent person. I checked out his LinkedIn profile, and I didn’t know anything about SPAM, law school, business school, being an entrepreneur, or how the Internet works. The folks in Peace Corps China always talked about being resourceful, so I looked for and found an opportunity to connect with him on a level that I could grasp. Matthew, who has unbelievable credentials and professional accolades, still has “Ski Instructor” on his LinkedIn profile somewhere between “Adjunct Professor of Law” and “Harvard Business School”:
I had just spent all of my time in China aiming to build relationships with my students and other people in my community that were from vastly different backgrounds and trying to find common ground from which to build rapport and trust. I thought, if someone this accomplished keeps their ski instructor experience on their resume, it must have a lot of meaning. I’m glad I followed that intuition because this topic led to a great conversation with Matthew about hometowns, ski trips, and ski equipment, which eventually lead to a conversation about surfing and surfboards, which is right in my wheelhouse. It turned out to be a great interview because we connected over things that we both found important. We found a piece of common ground that didn’t seem obvious at first—part of that being a deep curiosity for how and why things work. Looking back five years, I can say without reservation that finding a way to connect with Matthew that day has had a profoundly positive impact on the course of my life.
When it came time for me to interview with our co-founder, Michelle, she understood that I had a lot to learn about the company, and she took the time to draw out a simplified map of Cloudflare’s network on a yellow legal pad. She drew jagged, little clouds around the world and patiently explained what global caching is, how Anycast networking helps with DDoS attacks, and how DNS is like the phone book of the Internet. I was struck that such a highly intelligent person, HBS grad, co-founder of a major tech firm would take time out of their busy day to do this. I learned later that Michelle is always like this. She is amazing with names, stops to talk to folks in the office whenever she can, and sets a tone of respect, compassion, and understanding at the office. It is inspiring.
I then had a video interview with John Graham-Cumming, our CTO, who was in London. There was no getting away from tech with this interview. So I Googled everything I could about John. I read his book Geek Atlas, I watched his TED Talk, and I looked into his interest in Movie Code. I was ready for this interview. We talked about the Parkes Radio Telescope in Australia, Alan Turing, and about the code in the Matrix (thank you, Neo!). John is a fascinating speaker and a legend in the technology space. He is also kind and patient, and he never made me feel silly for not grasping technical concepts right away.
After 6-7 interviews over the following weeks, the feedback I got was that I was a good culture fit, I was hard-working and smart, but I just didn’t have the technical knowledge to do the job. That feedback seemed spot on, but I wasn’t going to let that hold me back. I knew I could be useful to this company. I knew that if they gave me a shot and threw me into the deep end that I would learn to swim. I knew what I needed to do: learn the language and culture of Silicon Valley, make connections, and be resourceful.
I stood outside of the old Cloudflare office at 665 3rd St. in San Francisco, and I told myself that I have to get in that door. I didn’t know exactly what they are doing in there, but it seemed weird and interesting, and I wanted to be a part of it.
So I started learning. Another returned Peace Corps volunteer that I’d met in the Bay Area sat down with me one weekend and helped me build a simple website from the ground up. In the most basic HTML and CSS, we embedded a video we made about my China experience. On the site I made the background color orange to match the Cloudflare logo and wrote something like, “Check it out Matthew and Michelle, I’m learning how to write code!”, and I sent them the link.
In the following weeks, I sent more follow up emails to Matthew than felt polite. But it worked. Matthew, Michelle, and John took a huge risk on me, and I got an offer to be Cloudflare’s “Writer” (since that was really the only thing that made sense for an academic philosopher to do at a tech firm). They actually gave me business cards that read: Andrew A. Schafer – Writer.
When I accepted the offer via email, Matthew wrote back saying that getting up to speed with Cloudflare was “going to be like drinking from a fire hose”.
Drinking from the Fire Hose:
On day one, I sat down next to the folks on the Data Team and introduced myself. They all said a quick, polite “hi” and then put their head phones back on immediately and continued to write code. I didn’t learn for a long time that engineers DO NOT like to be interrupted when they are coding. This is a key feature of tech culture.
At one point John Graham-Cumming walked past my desk and asked me why I was staring at that man in the orange shirt so much. I turned around and exclaimed, “John, did you know that the Internet has LED lasers that blink on and off BILLIONS of times per second?!” He calmly replied, “yes” and then went about his business. That fact made my mind melt. I had so much to learn.
One of the first things I worked on as Cloudflare’s Writer was some of the PR efforts surrounding Project Galileo, DDoS attack protection for at-risk public interest websites, which I’m still proud of. I worked with our legal team to draft up this blog post, which helped me to understand the implications and power of Cloudflare’s technology in real-world terms.
I worked with Nick Sullivan a whole bunch at the beginning also, which was mystifying. He is already a great writer and he was writing about such complex things. There were times where I was adding punctuation to sentences that made sense grammatically, but I didn’t understand their content. I learned a lot about encryption, and my tech vocabulary grew.
At one point I also helped John Graham-Cumming with a few blogs. John is a published author, so I didn’t really help him write anything, but I did help him bring his posts way down to my level. You can see my influence on this blog post about Shellshock. That day I learned the term “zero day vulnerability”.
In that blog John wrote: “Attackers will also use an ACE vulnerability to upload or run a program that gives them a simple way of controlling the targeted machine. This is often achieved by running a “shell”. I read his draft and I asked him, “What is a shell?”. A question, I learned much later, that was highly embarrassing to ask at a tech office. But I didn’t know, and I wanted to know. So we clarified that, “A shell is a command-line where commands can be entered and executed” in the post just in case other tech noobs like myself were trying to follow along. I learned how to be a translator from tech-speak to normal English.
I even researched and wrote a few posts of my own, like this one about Raspberry Pi’s fronted by Cloudflare. I had no idea what a Raspberry Pi was before being asked to write this. Thankfully one of the folks on the Data Team had one and let me borrow it for a photo op. I learned about the inspiring philosophy behind Raspberry Pi and the vibrant community that uses them.
As the official Cloudflare Writer, I was proud of writing the copy for our dashboard. That project was an amazing way for me to get to know a lot of key members of the engineering team and have them teach me exactly how each feature worked. I wrote out what I understood, clarified some points with them, and then made a pull request to get the explanations into the code base for our dashboard.
If you’ve ever used these help menus—you are welcome! (Note: lots of other Cloudflare team members have kept this updated and expanded.)
Eventually, I became an honorary member of the Data Team. It took some doing, but I learned Python the hard way, and I wrote a Python script that would print my name 100,000 times in the terminal. I crashed my machine when I tried to make it print my name 100,000,000,000,000 times. I learned something about code that day—it can break things.
I ran this code while sitting next to the person who had built Cloudflare’s original database. I did a victory dance when I crashed my laptop I was so proud of myself. That is sort of like me bragging about my backyard badminton skills next to Serena Williams.
I dipped my toes into the language of code, and started to speak that language with the engineers around me. This helped me to learn an important lesson about tech culture: the deeper your technical understanding the greater the respect you receive.
Eventually, I was ready for a new challenge at Cloudflare—talking to our clients.
The first thing I learned in a client facing role at Cloudflare is that Cloudflare is not a widget or a nice-to-have—it is mission critical technology for everyone that uses it. When something goes wrong people are very upset. The second thing I learned in a client facing role at Cloudflare is that the Internet is a fragile little teacup and it runs on human trust—which is astonishing. The combination of those two facts created ample opportunity for me to develop my listening and communication skills.
I started by rereading How to Win Friends and Influence People, by Dale Carnegie and took special note of rule number four, which states, “Be A Good Listener”. I quickly graduated to the philosophy and practice of Nonviolent Communication, by Marshall B. Rosenberg. I ended up taking some NVC courses in San Francisco focused on listening skills in this style. I also took compassion meditation courses via Stanford a few years in a row, which had a profound impact on my ability to empathise with our clients.
While brushing up on and honing these interpersonal skills was helpful, what I learned in a lot of those early meetings with clients was that I need to understand Cloudflare’s technology better. It’s one thing to be able to talk about it, it’s a whole different thing to be able to understand it enough to solve real issues.
I decided to do the “homework” our Solutions Engineering team gives out to their hiring candidates. I had to learn command-line basics, create an origin web server on DigitalOcean, install Ubuntu, configure a firewall, install NGINX, create a simple website from HTML, add an image to that site, set up DNS, and then put Cloudflare in front of it.
I set up my first DNS record in Cloudflare to point to my origin server, and was like “OHHHHHHH SNAP! That is how DNS works! It maps my domain name to the IP address of my server!” Hands on learning makes all the difference.
And I learned that WWW is a subdomain of the apex!! What???
It wouldn’t be a legit Cloudflare blog without more code, so here we go. I ended up writing (modifying) this amazing piece of code based on the NGINX HTML welcome page template:
Notice that I added an image:
I’m now a web developer! I’ve added yet another cat photo to the Internet. You are welcome world! (Note at the time of publishing my site is offline [I forgot to renew the domain—oopsy]).
Once I had my site up and running on Cloudflare, I learned how to make API calls to pull down the our Enterprise raw logs and use jq to sort them (jq, I learned, is “a lightweight and flexible command-line JSON processor”):
(Note: This cURL command does not contain a real API key. I learned the hard way to NEVER include the API key when sharing a cURL.)
I was so proud. I could say things like, “pull down the raw logs and pipe them into jq” to my clients, and I actually knew what I was saying—my tech language skills were improving.
I then read “High Performance Browser Networking” by Ilya Grigorik. I didn’t even understand what that title meant at first. I had to translate it into non-tech English. It turns out that, for example, Chrome is a high performance browser, which is a tool you use to navigate a network of computers, a.k.a. the Internet. So it is a guide book for building the most performant web apps within the limits of current browser and networking technology.
Grigorik’s philosophy resonates with me, “Good developers know how things work. Great developers know why things work.” Insert any other profession or art and the statement remains true.
It took me six months of reading on bus rides to work, but by the end I could say things like, WebSocket API, Subprotocol Negotiation, TLS OCSP Stapling, and TCP Head-of-Line Blocking. I learned from Grigorik that, “TCP provides the abstraction of a reliable network running over an unreliable channel, which includes basic packet error checking and correction, in-order delivery, retransmission of lost packets, as well as flow control, congestion control, and congestion avoidance designed to operate the network at the point of greatest efficiency. Combined, these features make TCP the preferred transport for most applications.” Who knew?
After putting so much work into learning what Cloudflare really does, I came to understand something fundamental about the tech world: the learning never stops. Never. The fire hose never turns off.
When I started at Cloudflare we offered more domains and extra SSL cert hosting slots as our additional products. Now we have Workers and Access and Argo and Argo Tunnel and Spectrum and Load Balancing and Stream and a Mobile SDK, and the list keeps growing. And we all have to learn about all of this new technology as it gets released. It is amazing!
Over the last few years, I’ve learned the language of Silicon Valley, and more specifically, I can speak the language of Cloudflare fluently. That has made a huge difference in my career.
Life @ 101 Townsend:
I’ve enjoyed a lot of successes at Cloudflare, but the one achievement I’m most proud of is creating the “Big Horse Award for Strong Work”.
The idea for this came directly from chapter 2 of How to Win Friends and Influence People: “Give honest and sincere appreciation”. I make it a point to tell the folks I work with that they are doing outstanding work every chance I get because the folks I work with really are doing outstanding work all the time, and they should know about it.
Maybe three years ago my best friend at Cloudflare sent me a message via HipChat that read something like: “Hey Big Horse, you check that Jira ticket yet?”. From that day forward I called everyone “Big Horse” on HipChat at all times, which I thought was hilarious and everyone else thought was weird or annoying.
Shortly after that, in an effort to step up my “Give honest and sincere appreciation” game, I started sending emails to the whole company pointing out the strong work our support team was doing in our Zendesk customer support tickets. Our support team is world-class, but since only a few teams in the office can access Zendesk, a lot of folks internally don’t see their amazing work. I decided to take screenshots of tickets that were particularly well-handled and share them. I’d titled these emails “Strong Work, Big Horse!”. I quickly learned that emailing the whole company “does not scale”.
This culminated at one of our all hands B.E.E.R. meetings, where I gave out a Big Horse Award to a few outstanding members of our Support team. I had this stunningly beautiful trophy made for the occasion:
We needed a logo, so I Googled “stupid horse drawings” and found an image. With a little editing via photo editor and PowerPoint, a meme was born:
Since then we’ve had all sorts of iterations of the Big Horse logo:
And we had paraphernalia made:
Our support team even spray painted “Big Horse” on the side of a building on 4th St in downtown San Francisco on a team outing:
We’ve issued a new Sparkle Lama award as well—since not everyone wants to be called a big horse:
Many Cloudflare team members have Big Horse and Sparkle Lama stickers on their laptops, and we’ve shipped those golden big horse trophies around the world to our London and Singapore offices. These symbols have become easy ways to let our teammates know that they are doing great work. It is a small thing, but it adds up and helps make Cloudflare a great place to work.
Just a few weeks ago this Tweet was pointed out to me:
Well, Neil, the reason for this is that a few engineers and myself had big plans of launching a website around the Big Horse Award, we bought big.horse and a few others, but we didn’t follow through—yet. Stay tuned.
The Big Horse and Sparkle Lama Awards are my contribution the tech culture I’ve been a student of these last few years.
回中国 （Back to China)
Five years after those first conversations with Matthew, Michelle, and John, I’m headed back to China with Cloudflare!
We are expanding our presence in China, and I have the good fortune (幸福) to combine the skills I acquired in philosophy and in the Peace Corps with the skills I acquired in Silicon Valley. We will be onboarding new Chinese clients, hiring more team members, and building out partnerships with other Chinese tech firms. I’m incredibly lucky to be headed back to a country that I love and embark on a new adventure.
I have a whole new fire hose aimed at me, and I plan to drink deep. I’ve been taking Mandarin classes again, this time to learn words like encryption (加密), caching (缓存), and cloud software (云软件). I’ll be learning a whole new interpersonal skill set around working with clients in China and across Asia. And since the office is just starting, this project will be a new exercise in resourcefulness.
life_journey = ["China", "Silicon Valley", "China"]
for x in life_journey
I had no idea how much opportunity lay before me when I walked in the door as “the writer”, and I am profoundly grateful that Cloudflare took a chance on me. I plan to throw myself into this project in China, to learn and grow and contribute, and to figure out the best way to translate “Strong Work, Big Horse” into Mandarin.
A research group at NATO’s Strategic Communications Center of Excellence catfished soldiers involved in an European military exercise — we don’t know what country they were from — to demonstrate the power of the attack technique.
Over four weeks, the researchers developed fake pages and closed groups on Facebook that looked like they were associated with the military exercise, as well as profiles impersonating service members both real and imagined.
To recruit soldiers to the pages, they used targeted Facebook advertising. Those pages then promoted the closed groups the researchers had created. Inside the groups, the researchers used their phony accounts to ask the real service members questions about their battalions and their work. They also used these accounts to “friend” service members. According to the report, Facebook’s Suggested Friends feature proved helpful in surfacing additional targets.
The researchers also tracked down service members’ Instagram and Twitter accounts and searched for other information available online, some of which a bad actor might be able to exploit. “We managed to find quite a lot of data on individual people, which would include sensitive information,” Biteniece says. “Like a serviceman having a wife and also being on dating apps.”
By the end of the exercise, the researchers identified 150 soldiers, found the locations of several battalions, tracked troop movements, and compelled service members to engage in “undesirable behavior,” including leaving their positions against orders.
“Every person has a button. For somebody there’s a financial issue, for somebody it’s a very appealing date, for somebody it’s a family thing,” Sarts says. “It’s varied, but everybody has a button. The point is, what’s openly available online is sufficient to know what that is.”
This is the future of warfare. It’s one of the reasons China stole all of that data from the Office of Personal Management. If indeed a country’s intelligence service was behind the Equifax attack, this is why they did it.
Go back and read this scenario from the Center for Strategic and International Studies. Why wouldn’t a country intent on starting a war do it that way?
Gregory C. Allen at the Center for a New American Security has a new report with some interesting analysis and insights into China’s AI strategy, commercial, government, and military. There are numerous security — and national security — implications.
The New York Times and Reuters are reporting that China was behind the recent hack of Marriott Hotels. Note that this is still uncomfirmed, but interesting if it is true.
Private investigators looking into the breach have found hacking tools, techniques and procedures previously used in attacks attributed to Chinese hackers, said three sources who were not authorized to discuss the company’s private probe into the attack.
That suggests that Chinese hackers may have been behind a campaign designed to collect information for use in Beijing’s espionage efforts and not for financial gain, two of the sources said.
While China has emerged as the lead suspect in the case, the sources cautioned it was possible somebody else was behind the hack because other parties had access to the same hacking tools, some of which have previously been posted online.
Identifying the culprit is further complicated by the fact that investigators suspect multiple hacking groups may have simultaneously been inside Starwood’s computer networks since 2014, said one of the sources.
I used to have opinions about whether these attributions are true or not. These days, I tend to wait and see.
Back in October, Bloomberg reportedthat China has managed to install backdoors into server equipment that ended up in networks belonging to — among others — Apple and Amazon. Pretty much everybody has denied it (including the US DHS and the UK NCSC). Bloomberg has stood by its story — and is still standing by it.
I don’t think it’s real. Yes, it’s plausible. But first of all, if someone actually surreptitiously put malicious chips onto motherboards en masse, we would have seen a photo of the alleged chip already. And second, there are easier, more effective, and less obvious ways of adding backdoors to networking equipment.
In my book Data and Goliath, I write about the value of privacy. I talk about how it is essential for political liberty and justice, and for commercial fairness and equality. I talk about how it increases personal freedom and individual autonomy, and how the lack of it makes us all less secure. But this is probably the most important argument as to why society as a whole must protect privacy: it allows society to progress.
We know that surveillance has a chilling effect on freedom. People change their behavior when they live their lives under surveillance. They are less likely to speak freely and act individually. They self-censor. They become conformist. This is obviously true for government surveillance, but is true for corporate surveillance as well. We simply aren’t as willing to be our individual selves when others are watching.
Let’s take an example: hearing that parents and children are being separated as they cross the US border, you want to learn more. You visit the website of an international immigrants’ rights group, a fact that is available to the government through mass Internet surveillance. You sign up for the group’s mailing list, another fact that is potentially available to the government. The group then calls or e-mails to invite you to a local meeting. Same. Your license plates can be collected as you drive to the meeting; your face can be scanned and identified as you walk into and out of the meeting. If, instead of visiting the website, you visit the group’s Facebook page, Facebook knows that you did and that feeds into its profile of you, available to advertisers and political activists alike. Ditto if you like their page, share a link with your friends, or just post about the issue.
Maybe you are an immigrant yourself, documented or not. Or maybe some of your family is. Or maybe you have friends or coworkers who are. How likely are you to get involved if you know that your interest and concern can be gathered and used by government and corporate actors? What if the issue you are interested in is pro- or anti-gun control, anti-police violence or in support of the police? Does that make a difference?
Maybe the issue doesn’t matter, and you would never be afraid to be identified and tracked based on your political or social interests. But even if you are so fearless, you probably know someone who has more to lose, and thus more to fear, from their personal, sexual, or political beliefs being exposed.
This isn’t just hypothetical. In the months and years after the 9/11 terrorist attacks, many of us censored what we spoke about on social media or what we searched on the Internet. We know from a 2013 PEN study that writers in the United States self-censored their browsing habits out of fear the government was watching. And this isn’t exclusively an American event; Internet self-censorship is prevalent across the globe, China being a prime example.
Ultimately, this fear stagnates society in two ways. The first is that the presence of surveillance means society cannot experiment with new things without fear of reprisal, and that means those experiments — if found to be inoffensive or even essential to society — cannot slowly become commonplace, moral, and then legal. If surveillance nips that process in the bud, change never happens. All social progress — from ending slavery to fighting for women’s rights — began as ideas that were, quite literally, dangerous to assert. Yet without the ability to safely develop, discuss, and eventually act on those assertions, our society would not have been able to further its democratic values in the way that it has.
Consider the decades-long fight for gay rights around the world. Within our lifetimes we have made enormous strides to combat homophobia and increase acceptance of queer folks’ right to marry. Queer relationships slowly progressed from being viewed as immoral and illegal, to being viewed as somewhat moral and tolerated, to finally being accepted as moral and legal.
In the end, it was the public nature of those activities that eventually slayed the bigoted beast, but the ability to act in private was essential in the beginning for the early experimentation, community building, and organizing.
Marijuana legalization is going through the same process: it’s currently sitting between somewhat moral, and — depending on the state or country in question — tolerated and legal. But, again, for this to have happened, someone decades ago had to try pot and realize that it wasn’t really harmful, either to themselves or to those around them. Then it had to become a counterculture, and finally a social and political movement. If pervasive surveillance meant that those early pot smokers would have been arrested for doing something illegal, the movement would have been squashed before inception. Of course the story is more complicated than that, but the ability for members of society to privately smoke weed was essential for putting it on the path to legalization.
We don’t yet know which subversive ideas and illegal acts of today will become political causes and positive social change tomorrow, but they’re around. And they require privacy to germinate. Take away that privacy, and we’ll have a much harder time breaking down our inherited moral assumptions.
The second way surveillance hurts our democratic values is that it encourages society to make more things illegal. Consider the things you do — the different things each of us does — that portions of society find immoral. Not just recreational drugs and gay sex, but gambling, dancing, public displays of affection. All of us do things that are deemed immoral by some groups, but are not illegal because they don’t harm anyone. But it’s important that these things can be done out of the disapproving gaze of those who would otherwise rally against such practices.
If there is no privacy, there will be pressure to change. Some people will recognize that their morality isn’t necessarily the morality of everyone — and that that’s okay. But others will start demanding legislative change, or using less legal and more violent means, to force others to match their idea of morality.
It’s easy to imagine the more conservative (in the small-c sense, not in the sense of the named political party) among us getting enough power to make illegal what they would otherwise be forced to witness. In this way, privacy helps protect the rights of the minority from the tyranny of the majority.
This is how we got Prohibition in the 1920s, and if we had had today’s surveillance capabilities in the 1920s, it would have been far more effectively enforced. Recipes for making your own spirits would have been much harder to distribute. Speakeasies would have been impossible to keep secret. The criminal trade in illegal alcohol would also have been more effectively suppressed. There would have been less discussion about the harms of Prohibition, less “what if we didn’t?” thinking. Political organizing might have been difficult. In that world, the law might have stuck to this day.
China serves as a cautionary tale. The country has long been a world leader in the ubiquitous surveillance of its citizens, with the goal not of crime prevention but of social control. They are about to further enhance their system, giving every citizen a “social credit” rating. The details are yet unclear, but the general concept is that people will be rated based on their activities, both online and off. Their political comments, their friends and associates, and everything else will be assessed and scored. Those who are conforming, obedient, and apolitical will be given high scores. People without those scores will be denied privileges like access to certain schools and foreign travel. If the program is half as far-reaching as early reports indicate, the subsequent pressure to conform will be enormous. This social surveillance system is precisely the sort of surveillance designed to maintain the status quo.
For social norms to change, people need to deviate from these inherited norms. People need the space to try alternate ways of living without risking arrest or social ostracization. People need to be able to read critiques of those norms without anyone’s knowledge, discuss them without their opinions being recorded, and write about their experiences without their names attached to their words. People need to be able to do things that others find distasteful, or even immoral. The minority needs protection from the tyranny of the majority.
Privacy makes all of this possible. Privacy encourages social progress by giving the few room to experiment free from the watchful eye of the many. Even if you are not personally chilled by ubiquitous surveillance, the society you live in is, and the personal costs are unequivocal.
Earlier this week, the New York Timesreported that the Russians and the Chinese were eavesdropping on President Donald Trump’s personal cell phone and using the information gleaned to better influence his behavior. This should surprise no one. Security experts have beentalkingabout the potential security vulnerabilities in Trump’s cell phone use since he became president. And President Barack Obama bristled at — but acquiesced to — the security rules prohibiting him from using a “regular” cell phone throughout his presidency.
Three broader questions obviously emerge from the story. Who else is listening in on Trump’s cell phone calls? What about the cell phones of other world leaders and senior government officials? And — most personal of all — what about my cell phone calls?
There are two basic places to eavesdrop on pretty much any communications system: at the end points and during transmission. This means that a cell phone attacker can either compromise one of the two phones or eavesdrop on the cellular network. Both approaches have their benefits and drawbacks. The NSA seems to prefer bulk eavesdropping on the planet’s major communications links and then picking out individuals of interest. In 2016, WikiLeaks published a series of classified documents listing “target selectors”: phone numbers the NSA searches for and records. These included senior government officials of Germany — among them Chancellor Angela Merkel — France, Japan, andothercountries.
Other countries don’t have the same worldwide reach that the NSA has, and must use other methods to intercept cell phone calls. We don’t know details of which countries do what, but we know a lot about the vulnerabilities. Insecurities in the phone network itself are so easily exploited that 60 Minutes eavesdropped on a US congressman’s phone live on camera in 2016. Back in 2005, unknown attackers targeted the cell phones of many Greek politicians by hacking the country’s phone network and turning on an already-installed eavesdropping capability. The NSA even implanted eavesdropping capabilities in networking equipment destined for the Syrian Telephone Company.
Alternatively, an attacker could intercept the radio signals between a cell phone and a tower. Encryption ranges from very weak to possibly strong, depending on which flavor the system uses. Don’t think the attacker has to put his eavesdropping antenna on the White House lawn; the Russian Embassy is close enough.
The other way to eavesdrop on a cell phone is by hacking the phone itself. This is the technique favored by countries with less sophisticated intelligence capabilities. In 2017, the public-interest forensics group Citizen Lab uncovered an extensive eavesdropping campaign against Mexican lawyers, journalists, and opposition politicians — presumably run by the government. Just last month, the same group found eavesdropping capabilities in products from the Israeli cyberweapons manufacturer NSO Group operating in Algeria, Bangladesh, Greece, India, Kazakhstan, Latvia, South Africa — 45 countries in all.
These attacks generally involve downloading malware onto a smartphone that then records calls, text messages, and other user activities, and forwards them to some central controller. Here, it matters which phone is being targeted. iPhones are harder to hack, which is reflected in the prices companies pay for new exploit capabilities. In 2016, the vulnerability broker Zerodium offered $1.5 million for an unknown iOS exploit and only $200 for a similar Android exploit. Earlier this year, a new Dubai start-up announced even higher prices. These vulnerabilities are resold to governments and cyberweapons manufacturers.
Some of the price difference is due to the ways the two operating systems are designed and used. Apple has much more control over the software on an iPhone than Google does on an Android phone. Also, Android phones are generally designed, built, and sold by third parties, which means they are much less likely to get timely security updates. This is changing. Google now has its own phone — Pixel — that gets security updates quickly and regularly, and Google is now trying to pressure Android-phone manufacturers to update their phones more regularly. (President Trump reportedly uses an iPhone.)
Another way to hack a cell phone is to install a backdoor during the design process. This is a real fear; earlier this year, US intelligence officials warned that phones made by the Chinese companies ZTE and Huawei might be compromised by that government, and the Pentagon ordered stores on military bases to stop selling them. This is why China’s recommendation that if Trump wanted security, he should use a Huawei phone, was an amusing bit of trolling.
Given the wealth of insecurities and the array of eavesdropping techniques, it’s safe to say that lots of countries are spying on the phones of both foreign officials and their own citizens. Many of these techniques are within the capabilities of criminal groups, terrorist organizations, and hackers. If I were guessing, I’d say that the major international powers like China and Russia are using the more passive interception techniques to spy on Trump, and that the smaller countries are too scared of getting caught to try to plant malware on his phone.
It’s safe to say that President Trump is not the only one being targeted; so are members of Congress, judges, and other senior officials — especially because no one is trying to tell any of them to stop using their cell phones (although cell phones still are not allowed on either the House or the Senate floor).
As for the rest of us, it depends on how interesting we are. It’s easy to imagine a criminal group eavesdropping on a CEO’s phone to gain an advantage in the stock market, or a country doing the same thing for an advantage in a trade negotiation. We’ve seen governments use these tools against dissidents, reporters, and other political enemies. The Chinese and Russian governments are already targeting the US power grid; it makes sense for them to target the phones of those in charge of that grid.
Unfortunately, there’s not much you can do to improve the security of your cell phone. Unlike computer networks, for which you can buy antivirus software, network firewalls, and the like, your phone is largely controlled by others. You’re at the mercy of the company that makes your phone, the company that provides your cellular service, and the communications protocols developed when none of this was a problem. If one of those companies doesn’t want to bother with security, you’re vulnerable.
This is why the current debate about phone privacy, with the FBI on one side wanting the ability to eavesdrop on communications and unlock devices, and users on the other side wanting secure devices, is so important. Yes, there are security benefits to the FBI being able to use this information to help solve crimes, but there are far greater benefits to the phones and networks being so secure that all the potential eavesdroppers — including the FBI — can’t access them. We can give law enforcement other forensics tools, but we must keep foreign governments, criminal groups, terrorists, and everyone else out of everyone’s phones. The president may be taking heat for his love of his insecure phone, but each of us is using just as insecure a phone. And for a surprising number of us, making those phones more private is a matter of national security.
I’ve bloggedtwice about the Bloomberg story that China bugged Supermicro networking equipment destined to the US. We still don’t know if the story is true, although I am increasingly skeptical because of the lack of corroborating evidence to emerge.
We don’t know anything more, but this is the most comprehensive rebuttal of the story I have read.
BGP hacking is how largeintelligenceagencies manipulate Internet routing to make certain traffic easier to intercept. The NSA calls it “network shaping” or “traffic shaping.” Here’s a document from the Snowden archives outlining how the technique works with Yemen.
Bloomberg has another story about hardware surveillance implants in equipment made in China. This implant is different from the one Bloomberg reported on last week. That story has been denied by pretty much everyone else, but Bloomberg is sticking by its story and its sources. (I linked to other commentary and analysis here.)
Again, I have no idea what’s true. The story is plausible. The denials are about what you’d expect. My lone hesitation to believing this is not seeing a photo of the hardware implant. If these things were in servers all over the US, you’d think someone would have come up with a photograph by now.
Bloomberg is reporting about a Chinese espionage operating involving inserting a tiny chip into computer products made in China.
I’ve written about (alternate link) this threat more generally. Supply-chain security is an insurmountably hard problem. Our IT industry is inexorably international, and anyone involved in the process can subvert the security of the end product. No one wants to even think about a US-only anything; prices would multiply many times over.
We cannot trust anyone, yet we have no choice but to trust everyone. No one is ready for the costs that solving this would entail.
EDITED TO ADD: Apple, Amazon, and others are denying that this attack is real. Stay tuned for more information.
EDITED TO ADD (9/6): TheGrugq comments. Bottom line is that we still don’t know. I think that precisely exemplifies the greater problem.
I’ve previously written about people cheating in marathon racing by driving — or otherwise getting near the end of the race by faster means than running. In China, two people were convicted of cheating in a pigeon race:
The essence of the plan involved training the pigeons to believe they had two homes. The birds had been secretly raised not just in Shanghai but also in Shangqiu.
When the race was held in the spring of last year, the Shanghai Pigeon Association took all the entrants from Shanghai to Shangqiu and released them. Most of the pigeons started flying back to Shanghai.
But the four specially raised pigeons flew instead to their second home in Shangqiu. According to the court, the two men caught the birds there and then carried them on a bullet train back to Shanghai, concealed in milk cartons. (China prohibits live animals on bullet trains.)
When the men arrived in Shanghai, they released the pigeons, which quickly fluttered to their Shanghai loft, seemingly winning the race.
The company ProtectWise just published a long report linking a bunch of Chinese cyber-operations over the past few years.
The always interesting gruqq has some interesting commentary on the group and its tactics.
Lots of detailed information in the report, but I admit that I have never heard of ProtectWise or its research team 401TRG. Independent corroboration of this information would be helpful.
The collective thoughts of the interwebz
The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.