Network engineering is a world filled with jargon that sometimes sounds more like a secret language than a professional lexicon. Conversations with network engineers can be daunting for those outside the field, leaving many to feel lost in translation. Yet, our journey at ElastiFlow has unveiled the beauty and complexity of this domain, revealing invaluable lessons that extend far beyond the confines of technical expertise.
Lessons learned
Seeing through the Jargon
If you’ve ever paired with a network engineer for a bit, you may have heard a sentence like this: “We optimized the MPLS backbone for VoIP prioritization over SD-WAN using QoS and BGP, reduced jitter, and improved security and performance with ACLs, IPv6, and AI-driven IDS''. Yeah, I don’t know what that means either. However, if you look closely and ask questions, you’ll see that they are just speaking with a high level of specificity. Here are a few terms we’ve deciphered over the years:
Ingress & Egress
Imagine the internet as a bustling global shipping hub, with countless parcels of data zipping around. In this hub, network engineers are like the master logistics coordinators, deeply involved in managing not just any mailroom but a highly sophisticated data dispatch center. When they talk about ingress and egress, it's with an eye on efficiently routing these data parcels to optimize delivery speeds and ensure security.
Ingress: The Art of Receiving
To a network engineer, ingress isn't just about data arriving; it's about how effectively the network handles that incoming traffic. They meticulously configure ingress points (think of these as specialized mail slots) to ensure data is received, screened, and sorted. It involves setting up firewalls and intrusion detection systems to inspect incoming packets for threats, similar to how a mail service center scans incoming packages for hazardous materials. Ingress control also means applying rules to prioritize certain data types — ensuring that a live video stream gets a fast lane, much like expedited shipping for a critical package.
Egress: The Art of Sending
On the flip side, egress to a network engineer is about managing how data exits their network. This involves not just pushing data out but strategically determining the best paths for data to take to reach its destination efficiently and securely. Think of it as planning the optimal route for a delivery truck, avoiding traffic jams and road closures. Engineers use egress policies to limit bandwidth for non-critical data, akin to choosing ground shipping for less urgent parcels, while ensuring sensitive information is encrypted—like sending sensitive documents via registered mail.
ICMP
Imagine you're standing outside a friend's house, unsure if they're home. What do you do? You might shout, "Hey, are you there?" or perhaps send a text. In the digital world, when we want to check if a server is up and available, we do something similar: we ping it. Saying "ping" has become the digital equivalent of tapping someone on the shoulder to see if they're awake. It's the universal "Hello? Anyone home?" for computers.
But, when you step into the realm of network engineers, they get more specific. Instead of saying “ping,” they’ll interchangeably use a different term: ICMP, which stands for Internet Control Message Protocol. That might sound like someone decided to make saying "hello" as complicated as possible, but ICMP is essentially the formal attire of digital greetings. It's not just a casual "hey"; it's a structured way to ask, "Are you there, and if so, how quickly can you respond?"
Think of ICMP as sending a certified letter and requesting a receipt upon delivery but in the blink of an eye. You send a message and wait for a response, confirming the recipient is not only there but also how long it took them to reply. It's like playing digital echo in a vast canyon, where the echo's return time tells you how far away the other side is. ICMP is the protocol that governs these echoes, ensuring they're sent, received, and replied to in a manner that tells us something about the health and accessibility of a network.
So, when network engineers talk about using ICMP, instead of just saying "ping," they refer to the suite of tools and messages designed to diagnose network statuses. It's like saying, "I'm not just knocking on the door; I'm using a specific knock to see how sound travels through the house." In everyday digital life, "pinging" is a quick check-up call to a server, while ICMP is the underlying technology making sure your "Hey, are you there?" gets heard and answered with precision.
Packet Loss
Imagine sending a text message to your friend about your plans for the weekend. Now, imagine two scenarios: In the first, your friend acknowledges every message you send, confirming they've received it. If they don't, you resend the message until they do. This, in the world of network engineering, is akin to TCP (Transmission Control Protocol). It's the diligent, reliable friend who ensures every detail of your conversation is noticed, ensuring every packet of data sent across the internet arrives at its destination in order and without loss. It's like sending a precious package with tracked shipping, where every step of the journey is monitored and guaranteed.
Now, enter UDP (User Datagram Protocol), the carefree cousin of TCP. When you use UDP to send data, it's like sending a bunch of postcards on your vacation. You write your message, send it off, and hope for the best. There's no looking back, no checking if the postcard arrived or if it got lost in the mail. In network terms, UDP just sends data out into the digital ether without waiting around to see if it gets to where it's supposed to go. This is great for things where speed is more important than precision—like streaming your favorite live sports event. If a few frames of the game don't make it to your screen, it's not a big deal; the game goes on, and you're none the wiser. It's all about getting the content there fast, not perfect.
So, when we talk about packet loss, each protocol is a different story. With TCP, packet loss prompts a resend, like double-texting, to determine whether or not your first message got through. But with UDP, packet loss is more like that philosophical question about a tree falling in the forest; if the data doesn't arrive, and the application doesn't need it to function correctly, does it really matter? For things that require precision and order, TCP is your go-to. For everything else that values speed over accuracy, UDP is your free-spirited friend, spreading data far and wide with no regrets.
Jitter
Imagine you're watching your favorite series online, and despite having a high-speed internet connection (those impressive 50/Mbps you mentioned to your friends), the video keeps freezing, or the sound drops out. It's like expecting a smooth highway drive only to hit a stretch of road that's more pothole than pavement. This inconsistency in your "digital journey" is what network engineers call jitter.
Jitter occurs when the time it takes for packets of data (the tiny bits of information that make up everything we do online) to travel from one point to another varies too much. It's not just about speed; it's about how predictably that speed is maintained. In less technical terms, if our internet traffic were cars, jitter would be the difference between a smooth ride and one where you're constantly speeding up and slowing down.
Storage & Compute are commodities, but networks aren’t, and that’s ok.
We typically categorize hardware and software into "the big three":
Compute, storage, and networking.
Compute and storage have become heavily commoditized; you can easily purchase a working PC with terabytes of storage from any local shopping center. However, networking equipment like routers and switches are less commonplace in homes, reflecting their bespoke nature. This uniqueness of networking, like the role of FPGAs in computing environments, allows for optimization and significant performance advantages. Along with the specific hardware, networks are often highly unique. Given the distinct requirements of a company's network, network engineers are compelled, or even required, to adopt a more customized approach in their design and implementation. However, despite the unique hardware in the networking field, one standard reigns supreme: every device speaks the same language. This means tools like ElastiFlow’s Unified Flow Collector can leverage this common language to better spread observability on the network without creating bespoke solutions for every device on the network.
AI is the future, not the end, for networking.
It’s becoming an everyday conversation now. Everyone’s asking: “In 5-10 years, will AI do all the networking for us?” We don’t think so. Who has to train those models? If we’ve learned anything from network engineers, it is that their domain knowledge has so much depth. AI's role will augment engineers' work and be a valuable tool in their tool belt. Look at tools like GitHub co-pilot & OpenAI’s chatgpt. It’s become commonplace to accelerate your everyday tasks using these tools, allowing network engineers to focus on design and orchestration instead of menial tasks and configuration.
One-Way Mirrors
In the intricate world of network engineering, professionals possess a unique vantage point akin to a one-way mirror. They see the entire digital ecosystem—its flows, blockages, and potential breaches—with clarity often invisible to others. This unique perspective sets network engineers apart, granting them insights into the digital realm's ebb and flow that remain obscure to the layperson. Within this sphere, a fascinating divergence in philosophy and practice emerges, broadly categorized into classic and new-wave approaches, each with its own merits and challenges.
The Classic Approach
The classic school of network engineering is built on a foundation of tried-and-true methods and tools that have stood the test of time. These engineers view their networks as intricate machines whose gears they know by heart, often relying on legacy systems and protocols that have reliably served their purposes. This approach, however, can sometimes lead to what's known as the "swivel chair mentality," where engineers must pivot between disparate tools and systems to manage and troubleshoot their networks. This method, while effective, can be cumbersome and inefficient, potentially hindering quick adaptation to the rapidly evolving digital landscape.
Embracing the New Wave
Conversely, the new wave of network engineering is marked by an eager embrace of innovation and a keen eye for the gaps left by traditional tools. These engineers are characterized by their willingness to explore and integrate emerging technologies, such as advanced observability platforms, flow data collection (both sampled and unsampled), and AI-driven network management solutions. This forward-thinking approach seeks to abandon the swivel chair mentality, aiming for a seamless and cohesive toolkit that enables more agile and effective network management.
There's a lot more to observe
Never before have you been able to grasp all the information that is passing through your network. As you scale the devices on your network, you will always need more information. Today’s network environments are growing in such unprecedented complexity, making it harder and harder to troubleshoot our slowness.
There are over 7,300 vendor-specific fields from which data can be collected; cloud services are adding an extra layer of complexity to our lives, and due to the data explosion that COVID caused, our network environments are more complex than ever. The problem we are seeing is there aren't enough network engineers to combat this ever-growing complexity we have been seeing in recent years. On top of that - the network monitoring space has not been innovated in over 20 years!
In Conclusion
In a world where old-school wizards of networking wield their ancient tools against the sorcery of the new age, we find that the quest for knowledge is endless. As for AI taking over? Perhaps one day, but until then, we'll enjoy the journey, chuckle at the complexity, and maybe, just maybe, learn to speak a bit of this mystical language ourselves. After all, in the grand scheme of things, it's not just about keeping the network alive; it's about keeping the spirit of discovery (and a good sense of humor) alive as well. So, thanks to the ElastiFlow network engineers—the unsung comedians of the tech world. May your flows be swift and your connections stable, and we’ll continue learning from you along the way.