Fully Open Edge Cloud

Rapid.Space - 5G Edge Computing for Digital Resiliency

Jean-Paul Smets, Rapid.Space CEO, explains how Edge Computing can be used to support 5G networks for Digital Resiliency, to preserve telecommunication infrastructures and industrial activities in case of conflicts.
  • Last Update:2022-12-16
  • Version:001
  • Language:en

Available on Youtube at the following URL - https://www.youtube.com/watch?v=wbEeEd1MTs4

5G Edge Computing for Digital Resiliency

Introduction text:

Hi,

I am Jean-Paul Smets, CEO of Rapid.Space, a European-based edge cloud provider, and 5G infrastructure vendor.

Rapid.Space is a joint company between Nexedi, the creator of edge computing, and Amarisoft, the creator of the 5G virtual radio access network. Together, Rapid.Space, Amarisoft and Nexedi have included many of the largest listed companies in Canada, France, Germany, Italy, Japan, Sweden, the UK, the United States, etc.

References:

https://www.rapid.space 
https://www.nexedi.com 
https://www.amarisoft.com

Today, I'd like to talk about digital resiliency and how edge computing and vRAN can help mitigate risks. 

------------------------------------

We live today in a world of ever-increasing disorder, as well as an ever-increasing dependency on the cloud and smartphones. 

We believe we are strong. But we are actually weak.

The most immediate risk we face is legal or political.

For example, LinkedIn got blocked in Russia some years ago. WhatsApp is blocked in Dubai. Developers can not use GitHub in Iran. Using Google Analytics is illegal all over Europe because it violates privacy legislation. Wechat got banned in the US. Virtually everything non-Chinese, including Wikipedia, is banned in China.

Any cloud service we use today could be banned tomorrow or face export restrictions. This is what digital philosopher "Tariq Krim" calls the Splinternet. Building our infrastructure on top of cloud services seems more and more like raising a skyscraper on the wetland.

Yet, there is much worse already happening.

Let us suppose someone wants to attack a country. Due to our dependency on mobile radio telecommunications and the cloud, this has become increasingly easy by targeting our digital infrastructure.

Internet cables cut in the south of France

Internet cables cut in the south of France

Cut submarine cables or land cables. This is what just happened in France on  Oct 21, 2022, and in Egypt on Nov 29, 2022.

References:
- https://sundries.com.ua/en/in-france-large-underwater-internet-cables-were-cut-a-russian-submarine-is-suspected-of-sabotage/
- https://www.theregister.com/2022/11/30/seamewe5_cut_outage_apac_africa/

And this is what you get: virtually no Internet access.

Internet outage in Egypt

Internet outage in Egypt

But also no access to clouds such as AWS or Azure.

No access to critical applications hosted on the cloud.

No more instant messaging with WhatsApp, Line or Wechat.

No access to Google maps and Wikipedia.

In a country depending on another country's cloud, it is guaranteed chaos.

---------------------------------------------

Cables are just one of many "critical points of failure" that we depend on.

Another weak point in all our countries is the so-called "core network" of 4G or 5G radio networks.

Without a core network, smartphones no longer work. And because most if not all core networks of mobile operators are implemented using a centralized architecture, it is extremely easy to destroy all communications in a country by destroying a single software: the core network.

On July 12th, 2012, 26 million subscribers of Orange, the largest French 4G operator lost all network access for 13 hours because of a bug in a software upgrade.

References:
- https://www.itespresso.fr/panne-orange-mise-jour-logicielle-coeur-reseau-54951.html

Core networks have limited redundancy: in the best cases, if the main core network has been destroyed, only 2 copies are ready to take over. But that is not even sure because disaster recovery plans are poorly or not tested at all. And even if they were tested, destroying the three sites hosting the core network of a mobile national network is a realistic military target.

--------------------------------------------------

What stands for 5G and core network stands in the same way for the cloud and their management panel.

References:
- https://phoenixnap.com/blog/cloud-outage
- https://www.cnbc.com/2021/12/12/aws-outage-and-hashicorp-ipo-point-to-a-multicloud-future.html

In October 2021, Microsoft Azure suffered a disruption that took down virtual machine services for six hours. For the duration of the outage, many users were unable to deploy new VMs or update extensions. Basic service management operations (such as start, create, and delete) also led to errors.

This is what happens whenever the central management panel of a cloud, something also called an OSS/BSS, is no longer functioning well or has been destroyed.

National clouds, which are immune to submarine cable attacks, are not immune to attacks on their central panel. Destroying a country's national cloud through its central panel is another realistic military target. Once it is destroyed, it is no longer possible to allocate new services or to reconfigure running services. or even repair them. 

-------------------------------------------------------

Things can get even worse with cloud network downtime.

Reference:
- https://techgenix.com/7-biggest-cloud-outages-services-2021/

In 2021, AWS attributed 7 hours outage to a large connection activity surge, which overwhelmed networking devices. This resulted in delays and latency between the internal AWS networks, which had ripple effects on customer apps. 

National clouds are not immune to this type of attack either which can range from a BPG attack to an advanced persistent threat hidden in the operating system of the ASIC of network switches. What is quite unique with network incidents is that one network element failure can sometimes trigger a chain reaction that propagates to the whole network in a matter of minutes.

---------------------------------------------------

And of course, we can put down a country that still relies on the Windows operating system by spreading a virus.

There are countless stories of hospitals hit by Windows ransomware that stops all desktop PCs with severe consequences for patients. Just like hospitals,  a country that relies on Windows is at great risk of being hit massively.

Reference:
- https://www.nbcnews.com/tech/security/ransomware-attacks-hospitals-take-toll-patients-rcna54090

------------------------------------------------------

Another issue with the cloud is that it is often located in a handful of data centers.

For example, Facebook's infrastructure relies on only 11 data centers. Many national clouds only use 2 or 3 data centers.      

911 attack

Destroying all data centers at once is a plausible scenario that should not be ignored. September 11 coordinated attacks in the united states should act as a lesson for cloud infrastructure.

With technologies such as graphite bombs, there is not even a need for missiles, explosives, or fire to achieve destruction. A cloud of graphite propagated by the air cooling system is enough and does not kill anyone

Fire destroy datacenters

......can destroy datacenters

References:
- https://en.wikipedia.org/wiki/Graphite_bomb
- https://datacentrereview.com/2018/08/tokyo-aws-building-site-burns-in-fatal-fire-according-to-report/

Graphite bomb

Graphite bomb

--------------------------------------------------------

In summary, our digital societies can be brought down by destroying:

  • submarine cables,
  • 4G/5G core networks,
  • cloud panels,
  • network routing,
  • Windows PC,
  • Datacenters


These are a lot of possibilities. The probability that none of them happen in case of conflict is... zero.
The probability that we lose internet and cloud in case of conflict is... 100%.

Reference:
- https://handbook.rapid.space/RS-Presentation.SimpleRAN.Taipei?portal_skin=Slide#/11

----------------------------------------------------------

So, what are the serious, effective solutions to mitigate those risks?

-----------------------------------------------------------

Cloud bureaucracy

Cloud bureaucracy

Backed by intense lobbying of foreign cloud providers

OK, that was a bad joke.

But maybe not so far from reality.

So, what are the serious, effective solutions to mitigate those risks?

------------------------------------------------------------

In Rapid.Space, we've been doing edge computing since 2008. This is why some people consider us as one of the inventors of edge computing. At that time, our company was called "VIFIB", part of Nexedi group.

Edge computing created in 2008

Edge computing was created in the EU in 2008

Reference:
- https://www.reuters.com/article/urnidgns002570f3005978d80025773e004eb5e8-idUS70376855520100614

We created 2008 an open-source cloud operation management system called "SlapOS" which was designed from day 1 for a distributed architecture.

We were worried at that time that any country could be brought down by destroying a few data centers. 

Therefore, rather than a few data centers with thousands of computers each, SlapOS cloud could handle thousands of data centers with a few computers each. Datacenters could range from someone's home to an office or even an airplane.  

Edge computing was born.

SlapOS was supported in 2010 by the French government as part of a project called "Resilience UNG". 

Edge computing on JAL 777

Edge computing on a JAL 777.

We flew SlapOS on a  777 airplane of JAL in 2014. This was the first flying edge cloud in a commercial airliner.

Reference:
- https://twitter.com/smetsjp/status/532981298751897601

And we then applied edge computing to new businesses: manufacturing, autonomous drones and 5G telecom.

Each application of edge was the occasion to learn a lesson and progress towards resiliency.

Each application of edge also taught us how a country could resist in case of attack on its digital infrastructure.

--------------------------------------------------------------

Manufacturing

We deployed SlapOS to automate the provisioning of an ERP for the assembly of Peugeot cars in a factory of Stellantis in Algeria, or for Opel cars in the factory of Stellantis in Namibia.

RS Edge in Stellantis Factory

Rapid.Space edge in a Stellantis factory

In both cases, it was a requirement for the factory to be able to access their ERP including during the times when the Internet was not available. For example, in Algeria, Internet access is shut down during university admission exams, including in factories. This is to ensure that students can not search for answers online.

Therefore, the infrastructure hosting the ERP should be inside the factory, on-premises, or else the factory can not be operated from time to time.

But, at the same time, we need to automate provisioning and operation management of the factory's ERP and everything that goes with it such as CDN, database, etc... We need to automate backup on a remote site, and disaster recovery, upgrades, etc. All this without depending on an on-site system engineer that would be too costly or difficult to hire. This is where we need an automation technology like the cloud combined with an on-premise infrastructure.

This is what edge computing is: the automation of the cloud and the resiliency of on-premise servers.

What we also learned when we started deploying servers all over the world is that the Internet... is broken. We all know for example that Internet connectivity from and to mainland China can sometimes face huge congestion that makes it unusable. But what few people know is that the same congestion problems happen also between France and Russia, even before the Ukraine war. Or between Ireland and France. Or between Taiwan and Vietnam. To/from Turkey, Dubai, Iran, etc. The Internet is no longer well-connected. The idea that Internet access works between a factory and another factory, or between a factory and a cloud data center can no longer be taken for granted.

All of us believe that the Internet is something like on the left side of drawing (https://handbook.rapid.space/NXD-SlapOS.Experience?portal_skin=Slide#/9). In reality, some nodes are not reachable and the connection between other nodes is unstable, like on the right side of the drawing where we see one node left alone and a dotted link representing the congestion and a broken. 

Internet is broken

Internet is broken

This is why we created re6st, written "r.e.6.s.t.". It is software that we run on every cloud node and edge node and which tries to find a way to circumvent congestion on the Internet or reconnect nodes that were disconnected. With re6st, we can circumvent all the problems of connectivity that exist on the Internet. Servers in France, Japan, Algeria, and mainland China can reliably exchange information at any time.

Re6st is not useful only for international Internet transit. It is also useful at the national level. When the OVH cloud provider faced a routing incident on 11 Feburary 2016, the servers that Rapid.Space was still hosting at OVH at that time and remained reachable. re6st could find routes from and to OVH that could circumvent the failed routes of OVH. Rapid.Space service was not disrupted, unlike services that solely rely on OVH for networking. This is why a tool such as re6st can greatly help survive major network incidents, including those happening within a single country during bugs, war, cyber-attacks, or natural disasters.

References:
- https://re6st.nexedi.com/
- https://www.lemondeinformatique.fr/actualites/lire-une-panne-de-routage-a-paralyse-de-nombreux-sites-heberges-chez-ovh-63893.html

Rapid.Space edgePOD

Rapid.Space edgePOD

Today, all the knowledge we have acquired for 14 years with the deployment of SlapOS edge computing in factories, homes, or offices is encapsulated into a product called "Rapid.Space edgePOD. Rapid.Space EdgePOD is a small 1U server that can fit into a small rack cabinet typical of telecom, offices, or factories. All connectors are front facing: power, networking, I/O. As soon as the edgePOD starts, re6st creates connections with other edgePOD all over the world, forming a kind of indestructible mesh of edge nodes helping each other to survive any connectivity issue.

This mesh also reduces latency between the edge and cloud by finding better routes. For example, many default routes from Vietnam to Taiwan go through the US. But with re6st, faster routes, for example through Japan, are discovered automatically, thus reducing the latency between Vietnam and Taiwan.

Rapid.Space edgePOD Panel

Rapid.Space edgePOD

Reference:
- https://beremiz.org/

We have also included Rapid.Space edgePOD advanced features for industrial automation.

Hard real line with Linutronix's PREEMP_RT. 

High precision time with Linux PTP, the precision time protocol also known as IEEE 1588v2. Deterministic networking with TSN, the time-sensitive network based on Ethernet. 

We support OPC-UA standards to interconnect programmable logical controllers to the field bus of the factory. And we even run Beremiz, an open-source programmable logical controller fully compatible with IEC standards. Actually, Beremiz is the only PLC to implement 100% of the standard.

A single edgePOD can thus run workloads ranging from real-time industrial automation up to high-performance ERP, on-premise, while keeping a copy of its configuration and data on a remote site, automatically, or on a second edgePOD on the same site.

-----------------------------------------------------------------

Drones  ​​​

Rapid.Space, which is part of Nexedi group, is the first small business in Europe that was awarded the leadership of a military research project from the European Defense Fund.

Dronedge-E

Reference:
- https://ec.europa.eu/commission/presscorner/api/files/attachment/865730/EDIDP%20-%20DRONEDGE-E.pdf.pdf

The DRONEDGE E project has implemented a novel edge computing architecture based on SlapOS which can be applied to unmanned air systems as well as hybrid systems.

Rapid.Space edge on C-Astral drone

Rapid.Space edge on C-Astral drone

Reference:
- https://www.c-astral.com/en/unmanned-systems

We are running SlapOS and re6st inside drones, together with OPC-UA over IPv6 multicast for distributed navigation control. And this is enough to create a resilient, flying mesh of autonomous drones that can survive partial destruction or loss of contact with the ground.

What is interesting about this project is that it uses exactly the same technologies as those we use for industrial automation in Rapid.Space edgePOD: PREEMPT_RT, OPC-UA, re6st, etc. This is a kind of proof of the resiliency of our edge computing architecture.

Rapid.Space Edge On Olimex

Rapid.Space edge on Olimex

What is also interesting is that, in the case of drones, we are deploying SlapOS on a low-power ARM CPU by STMicro, with only 512 MB or 1 GB of RAM. This is very small compared to Rapid.Space cloud servers based on Intel or AMD CPU with 256 GB or 1 TB of RAM.

The board, provided as open-source hardware, is made by OIlimex in  Bulgaria. It costs less than 50 USD. Since it is open source, anyone can manufacture the same board anywhere in the world.

It is energy efficient. Drones need to fly long enough and need to be lightweight. We thus have to run the Olimex board with onboard small batteries. Low power is thus a must.

What we learned from the application of edge computing to drones is that it is mandatory for an edge operation management system to support different architectures beyond Linux and x86. Support of ARM, low power CPUs is a must. Support of microcontrollers might also be required in some cases. And the support of other operating systems such as QNX, FreeBSD, or NuttX might be extremely useful.

Reference:
- https://www.olimex.com/Products/OLinuXino/STMP1/STMP157-OLinuXino-LIME2/open-source-hardware

-------------------------------------------------------------

I would like to conclude with the Open Radio Station. The Open Radio Station is a very small 4G/5G base station and edge server that has the same length as my smartphone, which only weighs 2.4kg. It is very popular with police, security, and defense forces worldwide.

This single box does everything: a 4G/5G base station, a complete 4G/5G telecom network including the core network, an edge computing server to run local applications, an OSS/BSS for automated operation, re6st backhaul for network resiliency. It is essentially the combination of everything we described previously for manufacturing or for drones, combined with a 4G/5G software-defined radio.

Just like edgePOD, it can run essential applications on-premise and circumvent congestion on the Internet.

Just like with drones, it can create a mesh with other base stations and run on batteries or solar panels in case of electricity shortage. 

ORS and Starlink

Rapid.Space ORS and Starlink

The Open Radio Station is a kind of ultimate resiliency device.

Combined with satellite Internet access, it provides 4G/5G Internet anywhere in the world, even if submarine cables are cut.

It has a local core network and can support various forms of handover based on a distributed architecture. This eliminates the risk of the destruction of central core networks.

Its cloud panel is open source: anyone can create any number of copies on as many sites as needed. This eliminates the risk of losing the panel. And since it uses a zero-knowledge approach, there is no risk of sharing the panel database between untrusted users.

It uses re6st for networking. This eliminates the need for hardware routers and provides more flexibility to fight routing protocol attacks.

It runs Linux, not Windows, and can deliver progressive web applications to smartphones. This eliminates the risk of Windows ransomware.

And it can be distributed in every village or home. With tens of thousands of Open Radio Stations interconnected with each other or over satellite, a country can survive a brutal attack on its core networking infrastructure, in its data centers.

ORS test in Nicaragua

Let us watch a demo by Charlie Crumpton, Rapid.Space business developer in LATAM, who created in 3 minutes a 5G tactical edge in Managua, Nicaragua.

Reference:
- https://www.linkedin.com/posts/charlie-crumpton-7a5651120_5g-4g-vran-activity-7006728321426169856-yPtA/?utm_source=share&utm_medium=member_desktop

What is impressive is that the Open Radio Station can be used by… politicians themselves. Charlie used to work for a member of parliament in the UK. Setting up a 5G network has never been that easy.

The other great aspect is that the Open Radio Station still runs even if the backhaul is down. For example, if the satellite connection is brought down as happened in Ukraine in early 2022, the Open Radio Station will still work autonomously. And two Open Radio stations can form a mesh network together.

The Open Radio Station is also designed to survive trade restrictions and blockades. All its hardware design is open source. Anyone can copy it and build it locally. We are producing it now in France and in Taiwan. We expect to produce it in more regions soon.

The Open Radio Station does not depend on a single vendor. For now, we use Intel CPU but we could also use AMD, Hygon, Fujitsu, Ampere, etc. Unlike other solutions, everything in the ORS is done in software. 

Let us watch a video of the assembly. You will see how easy it is.

Reference:
- https://www.youtube.com/watch?v=5-Z5vQairKM

------------------------------------------------------

Conclusion

I would like to conclude with a few words on virtual radio access network technology, also known as vRAN.

SimpleRAN

Some people call us "Open RAN". Actually, we are not "Open RAN". We are "SimpleRAN" or, if you prefer, "Open vRAN". Our goal is to make good products that are simple, stable, and cost-efficient and that empower end users to build their own 5G edge computing architecture with full access to source code, hardware design, and operation procedures. We rely on Amarisoft pure software 5G stack which is the de facto world leader in carrier-grade vRAN software sales. 

Open RAN is facing delays at Deutsche Telecom

Open RAN is facing delays at Deutsche Telecom.

References:
- https://www.lightreading.com/open-ran/deutsche-telekom-hits-open-ran-delays---sources/d/d-id/782034
- https://rethinkresearch.biz/articles/mavenir-follows-parallel-into-layoffs-claims-just-open-ran-growing-pains/

Rapid.Space SimpleRAN for tier-1 telco
Rapid.Space SimpleRAN for tier-1 telco

On the other hand, SimpleRAN is progressing fast. It is now adopted by two tier-1 telecom operators.

Rapid.Space SimpleRAN for security

Rapid.Space SimpleRAN for security

It is adopted by defense and security. Airbus just won a 1 billion euro contract based on technologies part of the SimpleRAN ecosystem.

Reference:
- https://handbook.rapid.space/RS-Presentation.SimpleRAN.LATAM?portal_skin=Slide#/5

The "Open Radio Station" demonstrates SimpleRAN in 2.4kg.

If you need digital resiliency for your country.

If you want to survive submarine cable cuts, 5G core network outages, cloud panel outages, network routing outages, windows ransomware, and data center destruction, the Open Radio Station is the way to go.

First, use it as it is, produces it locally, and deploy an initial resilient digital 5G edge infrastructure. Then adapt it, extend it, and improve it. That's what open-source hardware is meant for. And you will achieve complete digital resiliency.