Transcript
Copyright ⓒ 2011, Juniper Networks, Inc. 1
CREATING INNOVATIVE
EMBEDDED APPLICATIONS IN
THE NETWORK WITH
THE JUNOS SDK
This work is an excerpt from the
forthcoming book “Network
Embedded Management and
Applications” expected to be
published in 2011 by Springer Press.
2 Copyright ⓒ 2011, Juniper Networks, Inc.
Creating Innovative Embedded Applications in the Network with the Junos SDK
Evolving a System into a Platform
Juniper Networks, Inc. values innovation and realizes the power of the software that runs the network. The company has always
taken pride in hardware and silicon innovation, building equipment that pushes the performance envelope of the industry. Its
chief software asset, on the other hand, is widely understood to be the Junos software. Junos software is a portfolio in its own
right: the Junos Operating System (OS), which runs the network elements; Junos Space, which runs across the network elements;
and Junos Pulse, which runs on the network endpoints.
The Junos OS is known as a system that runs reliably on top of any supported hardware, while having only one release train
and a consistent operational feel on any device. While customers value these traits, many do not know that since 2007 Juniper
Networks has opened this system to innovation by creating the Junos Software Development Kit (SDK). The Junos SDK underlines
the company’s emphasis on software and commitment to innovation through network programmability.
Network programmability itself, however, is not the goal, but the means towards greater innovation and an evolution of
the network. In addition to creating SDKs, Juniper Networks is evolving its software systems to serve as platforms that
will run network applications. The Junos SDK enables ease of development and reuse of components for collaboration, while the
underlying Junos OS provides security, robustness, and resiliency to failure, and a widespread platform for application
deployment.
Here we look at the Junos SDK and the kinds of applications that it enables. Components and applications can be built,
packaged, deployed, and finally run natively right on the devices like other native software processes. The Junos SDK provides C
and C++ APIs with POSIX compliance, a familiar environment to those with UNIX programming experience. Standard APIs such as
libc and STL are available, along with many other common libraries. The more remarkable parts are the Juniper Networks-specific
APIs pulled from the Junos OS that allow dynamic control of the device’s functions. These APIs evolve, but with compatibility in
mind: the Junos SDK’s tools, APIs, and other elements release regularly in step with the Junos OS. Applications made with the
Junos SDK benefit from the platform portability of the Junos OS. They may run on any device running the standard Junos OS,
and most Juniper Networks hardware products are qualified for such support.
Applications in the Junos Architecture
Junos SDK-built applications integrate with the Junos OS to create innovative solutions in several architectural areas of the device.
Where can you deploy an application, and how can you determine where it is best suited?
We present the Junos OS architecture at a high level and in detail where it pertains to building applications. We continue by
explaining how the architecture fits over the hardware, and what types of applications are best suited to the environments we
describe. The figure at the end of this section summarizes the main conceptual separations.
Partitions of the Junos Architecture and Hardware
The Junos OS is a single network operating system integrating routing, switching, and security. Most Juniper Networks hardware
platforms run the Junos OS, and most of these platforms support Junos SDK-built applications. The network operating system
design has a fundamental logical division into three elements: the control plane, the data plane, and the services plane. Here we
discuss them within the context of Junos and bring out points relevant to applications.
The Control Plane
The control plane’s primary role is to manage and control the behavior of the device, including the other two planes. It must
have a global view of the device hardware and software. It also manages the user interface. The control plane in a Junos OS-
based device runs on hardware called a Routing Engine (RE).
Many Juniper Networks products have REs that are hot-swappable physical modules in the chassis. There is often a chassis
option for a redundant RE backing up the master, as well. Later we shall discuss how the Junos SDK facilitates taking advantage
of this redundant RE in your applications. Whether the RE is a physical module or is integrated into a smaller device, a master RE
must always be present, as it is the primary location of the control plane software that controls the rest of the device.
The control plane’s key element is the Junos OS software. The basis of the Junos kernel comes from the FreeBSD UNIX operating
system, an open-source software system. This mature, general-purpose system provides many of the essential basic functions of
an operating system, such as the scheduling of resources. To transform it into a network operating system, Juniper Networks
engineers have extensively modified and hardened it for the specialized requirements of networking. The
Copyright ⓒ 2011, Juniper Networks, Inc. 3
Creating Innovative Embedded Applications in the Network with the Junos SDK
task of managing the device is shared among the Junos kernel, many Junos daemons, and some ephemeral utility-style tools
launched on demand. The daemons and tools that come bundled with the Junos OS are considered part of the Junos OS
platform. Junos SDK-built applications for the control plane resemble these daemons and tools. In fact, Juniper Networks employs
the Junos SDK APIs internally when building these applications, which then become part of the common platform and Junos OS
software bundle.
The Junos SDK provides the Routing Engine SDK (RE SDK) APIs and tools to build applications to extend the control plane
software on the RE. Because an RE is always present in any device, RE SDK-based applications are always deployable without the
addition of any extra hardware or software.
On the control plane, the Junos kernel and many Junos daemons expose APIs so that other processes can programmatically
manipulate their states and make use of their services in a dynamic way. The daemons that control the Junos user interface also
allow for programmatic and seamless extensibility of the user interface. New applications added to a device can thus be
configured and administered in the same ways as the Junos OS platform applications.
The Data Plane
The data plane spans many aspects of a device’s chassis and its modules. It is collectively referred to and abstracted as the
Packet Forwarding Engine (PFE) and is comprised of ASIC-based hardware and Junos OS microcode that performs packet
processing that is generally stateless. Aiming to perform at fast wire speeds and within its hardware resource limits, the PFE
generally delegates stateful packet processing to the services plane.
The data plane’s chief role is forwarding traffic according to the forwarding table, which is primarily formed through the routing
control service on the control plane. The data plane’s main extended abilities include switching, filtering, rate limiting, shaping,
and other quality-of-service (QoS) functions. These functions are originally controlled in software on the control plane, and much
of that software exposes APIs.
Today, Junos SDK applications do not run in the data plane, but an application running on the control plane or the services
plane can certainly influence packet processing in the data plane by using APIs. Fortunately for application developers, the details
of data plane hardware are abstracted out in APIs that are common across hardware platforms. Some APIs enable use of
features unique to specific hardware platforms.
The Services Plane
The services plane can be thought of as an extension to the data plane to perform stateful services and any services non-native
to the PFE. A classic example of a Junos service application is the stateful firewall.
In traditional forwarding devices, the data plane usually handles most of the packet processing, and the services plane runs on
optionally installable and hot-swappable hardware, which we generically name services modules. These contain one or more
service processing engines, which connect to the PFE at speeds up to 10 Gbps (circa 2010 hardware). The services plane includes
all services modules and engines in a chassis, and a given service application can be deployed on more than one engine or
module.
In security- and service-oriented devices, the services plane is the primary packet processor, and the data plane merely connects
a chassis mostly full of services modules. While these devices can perform forwarding, they are intended to service traffic in
stateful ways implemented in software running on the service engines across the services modules. This focus contrasts with the
devices mentioned above that are purpose built to forward packets in hardware, but the services modules are similar, and the
Junos SDK supports a common model for service application development in either case.
Each service engine runs a Junos OS kernel with real-time processing capabilities. The kernel is essentially the same as the one
present on an RE, but far fewer and different daemons run, since the service engine is geared towards servicing tasks instead of
the general device management done on the RE. Also, the hardware resources differ greatly from those on the RE. Each service
engine consists of a multiprocessing, multithreaded CPU and more memory than the typical RE, so that the software-based
services can quickly process packets in parallel and maintain a large amount of state. The Services SDK encompasses the APIs
and tools to build applications to run on the service engines.
4 Copyright ⓒ 2011, Juniper Networks, Inc.
Creating Innovative Embedded Applications in the Network with the Junos SDK
The Junos kernel on a services engine further logically divides packet handling for applications in two sub-planes. Its services
plane (data plane extension) is for fast customized transit traffic processing. Its control plane is for traffic termination with the
kernel’s IP stack.
The control side here is used to send or terminate traffic, as can be done on the RE or any usual host system. The control sub-
plane inside a service engine can be used to run a control-plane-style application that would normally run on the RE. Such
components frequently implement a server or signaling to communicate outside the device or simply to other components on the
RE. Co-location of such a control-plane-style component with a service-plane-style component benefits many applications because
the two components can share memory and information more easily and with greater performance.
Applications in the services sub-plane of a service engine can take on two roles involving inline packet processing: transforming
and monitoring. Transforming applications have access to any traffic flowing through the PFE that is selected for servicing at the
given service engine. They can modify, drop, hold, and create entire IP packets. Sending and receiving this service traffic by using
Services SDK packet-processing APIs happens in parallel and much more quickly than it would in a program using traditional
sockets. Monitoring applications work similarly, but the packets they receive are duplicates of some traffic originally transiting the
PFE. The original traffic is not impacted or serviced in this case, hence the term “monitoring.” A hybrid application can
differentiate and deal with both original and duplicated packets. Another kind of hybrid application, namely a gateway, may
combine a server or signaling control plane component with a transforming or monitoring services plane component. Many
familiar service applications, such as IPsec, work with both styles of components.
Integrated Hardware
An application built with the Junos SDK can run on either a routing engine (RE) or a services module. On some of the smaller
Juniper Networks devices, physical modules do not plug in to a chassis. Rather, a single enclosure contains the hardware
necessary to run a built-in control, data, and services plane. Nonetheless, applications still are supported in the control and
services planes, and we continue to use the routing engine and services module terminology.
Traffic Types
As applications take on different roles working in either the control or services plane, they deal with two different traffic types.
Control traffic is dealt with on the control plane. This traffic is either entirely internal to the device (e.g., inter-process
communication), or it is traffic that is destined to or sourced from an address of the device. Most addresses configured on the
device belong to the master RE. For example, all addresses configured on network interfaces for I/O ports and the loopback
interface pertain to the RE, so control traffic destined to those addresses is forwarded to the master RE and handled there. The
Junos OS also allows the configuration of addresses on interfaces representing a service engine. Traffic destined to those
addresses is forwarded to and handled on the given service engine’s control sub-plane.
Data traffic flows through the data plane. That which is related to any Junos SDK-built application is the data traffic selected for
servicing or monitoring. This is the traffic seen in the services plane handled by transforming or monitoring applications. Data
traffic would naturally flow through the PFE as transit traffic, but by special mechanisms it can be selected for steering through
the service engine’s services plane. Upon exit, it is rerouted and filtered in the PFE as if it were entering the device from any I/O
interface.
Copyright ⓒ 2011, Juniper Networks, Inc. 5
Creating Innovative Embedded Applications in the Network with the Junos SDK
Figure 1. Architecture summary and traffic paths
Working with Junos Features through the SDK APIs
The APIs of the Junos SDK allow you to make use of the features provided by the Junos OS platform software. Having
applications co-resident on the device naturally creates savings, but combining the existing value of the application with the ability
to utilize Junos OS features allows unprecedented customization of a carrier-class network operating system. The main features
covered in this section show you part of a growing body of platform functions that can be built into your application with the
APIs provided.
Familiar Basics
Given that the basis of the Junos kernel comes from the FreeBSD UNIX operating system, most C and C++ UNIX developers are
familiar with the API basics in the Junos SDK: the POSIX APIs, POSIX threads, the C standard library (libc), the C++ standard
template library (STL), socket programming, and syslog. Developers using the Junos SDK can immediately take advantage of this
knowledge, or even reuse existing code developed to these common standards. Third-party libraries of custom utilities that are
not already packaged in the Junos SDK are generally quick to port and use during application development as well.
One different but basic concept present in the Junos OS for which the Junos SDK provides APIs is that of routing instances.
These are analogous to virtual routers. Communication using sockets is enhanced by optional tying of the socket to a routing
instance. Routing instances normally are configured by the operator, but an internal instance also is reserved for all Junos SDK-
built applications to use for their own IP socket-based inter-process communication (IPC). The Junos SDK provides an IPC library
of APIs that uses these features for internal control traffic. It facilitates IPC on the RE and service engines, or between them,
without the need for IP addressing. Because this library automatically uses the internal routing instance reserved for new
applications, using it means that these applications cannot interfere with the Junos OS platform software’s IPC and other traffic in
the device. Applications also benefit from the inherent security of the internal routing instance: no connections from outside the
machine can be made to these sockets.
Working with Interfaces
The Junos OS primarily manages the chassis hardware with configured network interfaces. Some interfaces are internal as well.
Junos SDK APIs provide access to interface naming and properties, which are considered on four levels: device, logical, protocol
family, and addressing. Applications can register to be notified of interface state changes at each of these levels. For example, a
device interface could go down when the cable to the interface’s port is disconnected, and an event would be generated to the
registered applications. While the RE has a global view of all the interfaces in the device, the Services SDK applications are limited
to seeing only the information for the interfaces local to the service engine on which they run.
Routing Engines – Control PlanePacket Forwarding Engine – Data PlaneService Engine – Service PlaneServices(data)
planeServiceapplicationControl PlaneI/OModulesI/OModulesApplication(s)User interfaceextensionsIngressTra cEgressTra cLocal
control tra cInternal control tra cNon-serviced(transit) datatra cLocal controltra c addressedto service moduleServiced (transit) data
tra cBuilt with theRE SDKBuilt with theRE SDKBuilt with theServices SDK
6 Copyright ⓒ 2011, Juniper Networks, Inc.
Creating Innovative Embedded Applications in the Network with the Junos SDK
The interfaces that manage service engines can also be paired up in a redundant configuration. In such a mode, the redundancy
state and properties are entirely retrievable, and applications can listen for state changes such as switchovers.
In addition, all of the per-interface traffic and error statistics for local and transit traffic typically are accessed through the Junos
UI. RE SDK APIs, however, allow for an application to retrieve these interface statistics programmatically.
Manipulating the Data Plane Functions
Routes and filters traditionally are static and are configured by an operator. Dynamic route manipulation has been possible only
through the use of routing protocols. With the advent of the Junos SDK, routes and filters also can be manipulated quickly by
applications using Junos SDK APIs.This capability allows you to wield the data plane’s line-rate packet processing and scaling of
forwarding and filtering as you wish. The libraries responsible for this functionality work with the RE and Services SDKs, but both
routes and filters are managed by Junos OS platform daemons on the RE and eventually distilled into a forwarding table and set
of applied filters. These entities are sent to the packet forwarding engine for processing in hardware.
Properties of the forwarding tables and the table entries themselves are retrievable for any routing instance via the route-related
APIs. IP route manipulation in any table is also possible. One can programmatically create and delete routes with parameters
equivalent to those of a configured static route.
The internal process of route management in the Junos OS is complex, given the sophisticated nature of the supported routing
protocols, routing instances, and routing policy. Handling of these items is greatly simplified with Junos SDK APIs. These
programmatic abilities, combined with the interface information described in the previous section, allow you to develop a dynamic
routing protocol that is decoupled from the rest of the system.
Stateless firewall filters in the Juniper Networks devices can match packets based on layer 2 through layer 4 protocol header
fields. You can take an array of actions on packets that match such filter conditions. Filtering actions include accepting,
discarding, redirecting, sampling (for monitoring applications), statistics counting, logging, rate limiting, and changing QoS
parameters.
Filters and rate limiters are created to be applied on ingress and egress of network interfaces. Junos SDK APIs allow you to
manipulate, apply, and remove such filters and rate limiters programmatically. Furthermore, for the filters that count packets, the
statistics are equally retrievable with APIs.
Seamless User Interface Integration
Applications on Junos can be administered by several user interfaces, such as the CLI, SNMP, NETCONF, and the Junos OS to
Junos Space Device Management Interface. All interfaces but SNMP function in either an operational or a configuration mode.
Commands issued in the operational mode upgrade software, trigger events, or most often show status and statistics to allow
inspection of the system’s operation. Commands issued in the configuration mode modify the configuration database, which is
persistent across reboots. The syntax for the user interface commands follows a predefined hierarchical schema that can be
extended with the addition of RE-based applications packages.
The RE SDK provides a simple language to facilitate extending the schema for new operational commands. It allows defining the
syntax for commands and registering a new application as the handler. When the application receives a command, it may want to
return some resultant data. This return is facilitated by another new language used to define the structure and display format of
an XML response.
You can extend the schema for the hierarchy of object nodes in the configuration database. This capability allows the operator to
configure an RE SDK-built application with new syntax through any one of the Junos user interfaces. The same language used to
create new operational commands allows defining new configuration syntax for objects. It can also designate points within the
configuration database that, if present, require an application to be run, or if changed, require an application to be notified.
Through Junos SDK APIs, applications have read-only access to the configuration database. These APIs provide easy ways to
detect specific configuration additions, changes, or deletions, so applications can modify their behavior accordingly.
Write access to the configuration database and the ability to call configuration commands are not achieved through APIs. These
capabilities are left under the operator’s control, but with Junos scripting, the operator can run operational scripts to execute
commands and configuration changes. When an operator grants such a script executable permissions, it becomes executable
from an RE SDK-built application as well. Thus an application can effectively manipulate any aspect of the Junos system that is
exposed through the Junos user interface.
Copyright ⓒ 2011, Juniper Networks, Inc. 7
Creating Innovative Embedded Applications in the Network with the Junos SDK
Using Other Junos OS Features on the RE
You can develop innovative applications through use of functionality and features implemented in other components. On the
Junos OS platform, these reusable components could be the Junos daemons themselves, other Junos SDK-built daemons, and
even other Junos SDK-built libraries. Of course, the Junos SDK-built components need not all come from any one source: they
may come from multiple third parties creating software with the Junos SDK.
Here we give a few more examples of other features built into a wide array of applications shipped in the Junos OS platform.
SNMP
You can integrate with the Junos SNMP framework just as seamlessly as with the other user interfaces. Using the publically
available Net-SNMP APIs provided in the RE SDK, applications can register themselves as SNMP sub-agents. As such, they can
register new MIB objects and accordingly handle requests for information retrieval as these are directed to the application. They
can send SNMP traps as well.
AAA
The Junos authentication, authorization, and accounting (AAA) infrastructure has more features that applications can use through
the Junos SDK APIs. For instance, the operator may configure an LDAP or RADIUS server connection for the Junos OS to use for
AAA purposes. This connection is reusable through APIs that communicate through the proxy in the Junos OS to automatically
send authentication requests or accounting information to the configured server. This example shows how an application can
reuse functionality from another component rather than implementing a feature from scratch.
Packet Processing, Data Traffic Distribution, and Concurrency Control
One of the most important features of the Services SDK is that it facilitates the setup and processing of data traffic. Multiple
packet-polling loops, each running on a real-time thread, are dedicated to processing the data traffic in the services plane. The
setup of this feature is quite flexible and controlled partially by the operator through the configuration and partially by the
developer specifying the number of real-time threads. More than twenty real-time packet-processing threads can be set up per
service engine. The configuration parameters and programmatic decisions control the amount of CPU resources dedicated to
packet processing and the degree of parallelism versus speed of processing. The packets that are sent and received using the
Services SDK APIs are delivered to the application as quickly as possible using zero-copy I/O from the kernel to the queues
polled by the user-space application.
Since there are many real-time threads, each with its own queue to poll for packets, the system needs to decide how to distribute
received packets among the many queues. This configurable property can be set one of two ways: round robin, or flow affinity.
The round robin method distributes the packets evenly, cycling through the queues and back again continuously. The flow
affinity method ensures that packets of the same flow, or actually the same 3- or 5-tuple, always are distributed to the same
queue and hence to the same thread. The tuple defined for a flow usually consists of the protocol, along with source and
destination IP addresses, and optionally source and destination ports.
Real-time threads can be created with API extensions added to the normal POSIX thread library, but other Services SDK APIs
simplify this task even further to create and start the threads along with matching packet queues. Each real-time thread runs
without preemption on what we call a virtual CPU because the software thread is exclusively bound to a single hardware thread
of the underlying multi-processing CPU. While these threads are necessary for the packet loops, the Services SDK also provides
APIs allowing the creation of threads for generic real-time processing just as easily.
Although the POSIX thread-locking structures are available, real-time threads benefit more from the use of spinlocks and atomic
operations. The Services SDK provides simple APIs for both, optimized for the service engine environment.
Once an application holds a packet, it may use a comprehensive suite of APIs to access or change its metadata or actual content.
The metadata, for example, contains information such as the ingress interface and modifiable parameters such as the interface or
the routing instance to which the packet should be sent back when it exits the service engine. The entire packet content is
available, starting from the layer 3 (IP) header. APIs would be used to, for example, rewrite bytes 20 through 40 of a packet, or to
find out if a packet is original data traffic or if it has been duplicated for monitoring. If the application needs to do accurate
time-based calculations for packet events, it may make use of precise hardware timestamping APIs to read a cycle count from the
current CPU, or to read the time at which the hardware received the packet.
8 Copyright ⓒ 2011, Juniper Networks, Inc.
Creating Innovative Embedded Applications in the Network with the Junos SDK
Using Other Junos OS Features on the Service Engine
The Services SDK provides API access to improved memory constructs for custom use by the applications. They are internally used
by other accessible features such as the forwarding database and the policy database.
Two sets of memory management APIs provided in the Services SDK help in this multi-threaded environment designed for low-
latency packet processing. The first consists of a set of shared memory APIs that allocate and deallocate memory from a memory
pool that has a hard-wired TLB entry, thus guaranteeing no TLB misses upon memory accesses. This memory can be shared
among several processes if desired, but it is equally useful within the context of a single process because of the benefit of
consistently fast access times. On top of this shared memory pool, another set of caching APIs provides improves allocation and
deallocation times with a lockless design.
The forwarding database is an optionally configurable feature that makes use of this memory. If it is turned on, the IP routing
and autonomous system information is downloaded to the service engine and installed in this fast-access memory. Because it is
quickly accessible, it is suitable for use in the fast lookups in the inline processing path of the services plane, but it is also
accessible in the control sub-plane.
The policy database is another similar optionally configurable feature. It is also carved out of the fast-access shared memory pool.
The policy database is populated by the applications with their own custom-configured packet-processing rules. It is intended for
use during policy lookups during packet processing, and it is especially useful in construction of service applications, as
described later.
Development with Security and Reliability in Mind
The Junos SDK enables applications to run securely alongside the Junos OS platform’s base applications while taking advantage
of a wide range of its interfaces. First this section looks at security from multiple perspectives. We examine how the operator, the
Junos OS software, and the device are protected and managed, and equally how the application developer can use Junos
features to build additional security into an application. Lastly, we examine how applications can leverage the sophisticated
mechanisms built into the Junos OS for software and hardware reliability and fault tolerance.
Securely Deploying and Operating Applications
All Junos SDK-built software packages are easily installed in the same way as Junos OS packages, and thus installation benefits
from the intrinsic security of the existing process. When the software package originates from a third party, this process demands
explicit configured authorization of the software provider. This authorization is verified in an administrator-controlled list of
third-party providers. Naturally, the active system operator, if not an administrator, must have been granted permission to
perform any software installations.
The Junos operating system further guarantees software authenticity during installation, but also every time any process is
launched. This mechanism safeguards genuineness, tamper resistance, and trustworthiness of the source using well-known
public-key cryptography techniques involving the certification of the provider and a provider-generated signature that is not
forgeable. Certification of third-party providers is securely controlled by Juniper Networks, while tools in the Junos SDK make for
an uncomplicated process for a partner. Furthermore, signatures are generated and incorporated into the software package
along with the certificate automatically during the package build process.
Once an application has been authorized and authenticated successfully, it is executed in a non-root user-space resource-
controlled environment managed by the Junos OS. The kernel provides each process with the familiar protected memory
environment, while ensuring that the process acts within policy-defined resource and access limits. These limits are controlled on
several levels: limits global to the operating system, those that are provider-specific, those configured by the operator, and those
imposed by the application developer. The most restrictive values over all levels are applied. They can control features such as the
familiar UNIX rlimits and access to certain APIs and system calls, and they can give certain permissions normally associated with
the root user.
These mechanisms provide sensible assurances about Junos SDK-built software. Such programs cannot directly interfere or cause
harm to the native Junos OS software or other third party software. An application can, however, make changes with wide-
ranging impacts on the device operation. For example, you must take great care when wielding the power of routing and
filtering, so that resulting changes are in accordance with the expectations of the operator.
Copyright ⓒ 2011, Juniper Networks, Inc. 9
Creating Innovative Embedded Applications in the Network with the Junos SDK
Developing and Managing Additional Security and Resilience
You can build security and resilience into an application in several ways using the tools and APIs of the Junos SDK.
As we have seen, one level of the policy-defined resource limits imposed on Junos SDK-built applications is controlled by the
application developer. In fact, the developer must associate each process with a role having some permissions. Native roles have
predefined permissions, but new roles may also be defined and used to apply customized permissions. In addition to secure
implementation and coding practices, this is one practical way to guarantee by design that an application does not act outside its
intended limits.
We use the Junos health-monitoring framework to deal with failures in applications. This framework allows applications or even
individual threads to be monitored for health by simple echoing of periodic heartbeat messages. Using APIs, an application
developer can optionally register with the health-monitoring framework to control the frequency of these passive heartbeats and
specify an action in the event of a given number of missed echoes. Typically the action is to assume that a failure has occurred
and restart the application. Innate to the Junos OS is the ability to detect and report applications that fail several times (thrash)
upon restart within a short time period. You can also configure other actions such as RE or service engine switchover for critical
applications, enabling a redundant hot-standby environment.
Application High Availability
When a device is provisioned with redundant REs or service engines, the applications can take advantage of their redundancy,
providing varying degrees of application hot-standby support.
On the RE, you can turn on hot standby with graceful routing engine switchover (GRES). To enable high availability in an
application, you can optionally run the application on both the master (active) and backup (standby) REs. The application can
register for notification of RE switchover caused manually or by software or hardware faults. APIs built on top of the Junos SDK’s
IPC libraries allow state replication between the master and the backup application, so that the standby instance, upon
switchover, can pick up where the previously active instance had left off.
Simpler high-availability support is also available for RE-based applications when GRES is enabled. In this design the application
need not run on both REs simultaneously. It can make use of APIs to store opaque binary data in semi-persistent storage. While
GRES is enabled, these data stores are automatically replicated to the backup RE. Using this functionality, upon a switchover the
application is launched on the new master RE, and it can easily pull its data back out of storage to pick up where it formerly left
off. Furthermore, in the context of a single RE, if an application is restarted for any reason, it can equally resume its previous
state if desired.
Finally, the service engines themselves also support a mirrored redundant configuration. Rapid switchover behavior is all the more
critical to mitigate packet loss, so applications installed on service engines can be augmented to support hot standby. As when
running an application on two REs simultaneously, applications can use the same APIs to replicate state from the master to the
backup.
Building a Service Application
In this section we describe the design considerations in building a service application (one that processes transit data traffic in
the services plane) with the Junos SDK. We examine methods of steering the data traffic to a service engine, and we look at the
necessary architectural components of a complete solution. Finally, we look at the different approaches in the services plane for
receiving data traffic and building a service application.
Managing the Services Plane and Data Traffic Steering
As we have described above, the control plane plays a large role in managing the software of the data and services planes. To
this end, the control plane must work in three key ways to create a functioning service application.
First, service application software must be installed on targeted service engines. Based on configuration, a Junos OS daemon on
the RE pushes the software onto each service engine before it can be run, since the service modules have no disk. This step
could involve installing different software on different service engines, or the same software if scaling or redundancy is required.
On security- and service-oriented devices, all the service engines run the same software, eliminating configuration of applications
per service engine.
10 Copyright ⓒ 2011, Juniper Networks, Inc.
Creating Innovative Embedded Applications in the Network with the Junos SDK
Second, once the service software starts on a service engine, to the software must be configured with policies to apply, and it
may want eventually to report back information through the Junos UI. Policies in this context are simply the configuration
pertaining to the service application, especially its packet processing. The model used to achieve the coordination of policy and
reporting is to have a central management component (daemon) on the RE’s control plane. This daemon coordinates multiple
software instances in the case of multiple service engines, and it handles RE-specific functionality such as loading custom
configuration and processing changes. It also communicates the relevant policy information to the corresponding service
applications connecting to it. Communication can flow in the other direction as well if a service application, for example, wanted
to send statistics for eventual display in response to an operational command.
Third, the management component optionally controls the data plane to steer packets to the services plane for servicing on
service engines. We examine three of several approaches to doing so. These approaches are actually only pertinent to
forwarding-oriented devices, where the service plane is optional. In security- and service-oriented devices, the services plane is
always in the packet processing path, so no steering is implicit.
Routes are the simplest steering approach. A route to a service engine is a route like any other, containing a prefix against which
to match and a next hop; but the next hop is an interface managing a service engine, as opposed to an external address. Once
this route is installed in the forwarding table, packets matching it are redirected to the service engine. Variants on basic routing
are also possible. For example, equal-cost multipath routing and filter-based forwarding are both interesting options that can
include routes to service engines. Routes can also be controlled programmatically with the Junos SDK’s APIs.
The second option is to create and apply service sets to network interfaces on ingress or egress. A service set is a
conglomeration of an identifier, one or more services’ policies to be applied, and a service engine to which to redirect packets for
servicing. When a service set is applied to an interface, the data plane selects packets flowing through that interface as part of
the service set, and accordingly steers them to the specified service engine for servicing. The service application can retrieve and
correlate a packet’s service set identifier in order to apply the correct policy configured in the service set. When traffic is selected
for servicing at a network interface, the service set is said to be an interface-style service set. Other service set styles are equally
useful for steering data traffic to the service engine, but this style in particular allows the traffic to be filtered so that only a
subset of the traffic is steered towards the service engine for servicing. This filtering is done with service filters based on Junos
stateless firewall filters.
A third option uses the Junos OS sampling framework. This feature of the PFE duplicates packets according to a configured
frequency and sample length. All packets that enter the PFE’s sampling engine do so as a result of the application of a stateless
firewall filter. These duplicate packets typically are steered to a service engine for service processing. Given that the original
packet is forwarded without impact and is not modifiable by the service, sampling is well suited to monitoring-style services.
Service sets can also be combined with sampling when a service chain is needed, as described in the next section.
These options, in combination with other Junos features, make for a large array of possibilities in the design of a service solution.
The Design Models and Components
In the previous section we described the management component that coordinates the service applications running on the
service engine. This section introduces two more components of a service application and two models available for building them
with the Services SDK. An individual service engine supports service applications of only one model at a time. However, the
service engine supports any variation of non-packet-processing applications running in its control plane.
Autonomy and Polling Control Enabled by the Process Model
In this model, the service engine supports a sole service application consisting of a single two-component daemon, as in the
figure below. The control component communicates with the management component, usually via the IPC APIs. At a minimum, it
receives and stores the configured service policies, but sending statistics and status information is commonplace as well. The
control component is so named because it deals exclusively with control traffic, whether it be IPC or any communication outside
the device.
Copyright ⓒ 2011, Juniper Networks, Inc. 11
Creating Innovative Embedded Applications in the Network with the Junos SDK
The data component uses these stored policies and performs the servicing on the data traffic, hence its name. This component
spans many real-time software threads, each tied to a single and exclusively used hardware thread of the service engine’s CPU.
We call such a real-time thread executing a packet-polling loop a data thread. During the startup phase before these threads are
started, the Services SDK and kernel set up a series of zero-copy input and output first-in-first-out (FIFO) queue pairs, where
eventually one data thread is tied to and services one pair of queues in its loop. Generally the steps of each loop are to receive a
packet from the input queue if one is available, process it, and enqueue it in its output queue. Processing is based on either the
service policies stored in the memory shared with the control component or the application’s per-flow or per-session state.
Figure 2. Services SDK daemon and traffic flow
Part of the Services SDK provides the APIs that assist the setup of this multithreaded environment to work with the hardware
threads of the CPU. As described above, it also includes fast-access shared-memory APIs, locking constructs suitable to the
environment, and packet manipulation functions.
Service Chaining and Session Management Enabled by the Plug-in Model
The Services SDK provides the plug-in model to allow several data-traffic-oriented services to coexist and even cooperate through
an event framework within a single service engine. Plug-ins can send and receive both synchronous and asynchronous custom
events in a loosely coupled way, but this section focuses on the standard plug-in events. Furthermore, this model facilitates
maintaining the service policies in the policy database and accessing flow- or session-based state. Also, IP fragments are
reassembled automatically before processing.
Figure 3. Services SDK plug-in and data traffic flow
REPFEControlcomponentService EnginePacket FIFO Queues:Unserviced Local or Internal Control TracTransit Data TracUnserviced
Transit TracDatacomponentManagementcomponentData Threads• • •ControlcomponentService EngineDatacomponentTransit Data
Trac• • •• • •CDCDCD
12 Copyright ⓒ 2011, Juniper Networks, Inc.
Creating Innovative Embedded Applications in the Network with the Junos SDK
In this model, a Junos OS platform daemon managing the plug-ins starts and creates its own data component to run one or
multiple plug-ins linked into it as in the figure above. The plug-ins are implemented as shared libraries. Each has an entry
function that, when called, registers callback functions to serve as the control and data event handlers. Control events, called
from non-packet-processing real-time threads, initialize the plug-in and update policies originated from a management
component. Real-time data threads are managed by the Junos daemon managing the plug-ins. Each data thread internally polls
for packets and dispatches them as data events serially through a chain of plug-ins. This model works with the service set
steering method, which allows for a service order to be specified by the operator per service set, and this order ultimately
determines the order of packet delivery, and hence service processing.
With every packet of a session (based on the classic 5-tuple) dispatched as a data event, a per-session context to store custom
session state is delivered as supplementary metadata. This state expedites processing if plug-ins store policy decisions in the state
once they have been looked up for the first packet of a session.
This model has numerous advantages for some styles of service applications. First, if a service is flow or session based, the
framework sets up the packet-polling loops (data threads), provides a session context with every packet, and provides session
open and close events. Services from multiple providers (Juniper Networks and third parties) can run on a single service engine
to collaborate in a configurable order.
On the security- and service-oriented devices, the service engines always run in this plug-in model. On the forwarding-oriented
devices, while the service engines support both models, this plug-in model supports pairing two service engines together, where
one is standing by with replicated state information in the event of a failure. Overall, this model generally provides more features
and facilitates rapid portable development where session-based packet processing is the main focus.
Virtual Build Environment
The Junos SDK provides a Virtual Build Environment (VBE) that creates a comprehensive and consistent way to develop
applications in just about any operating system environment you choose. The VBE is, in actuality, a FreeBSD operating system
image that is executable inside VMware Player, which is freely available software that runs on a wide range of operating systems.
This approach allows for a clean separation of a portable and duplicable build environment. Samba comes pre-configured in the
VBE, so that sharing the file system with that of the native system is easy to achieve. Thus, developers are free to use any
development environment software they choose.
The APIs and build tools of the Junos SDK are built and tested at the same time as the rest of the Junos OS, and they will
continue to be enhanced in step with the Junos OS release schedule. Developers manage the APIs and tools in the VBE by
installing packages of these APIs and tools matching the Junos OS versions for which they wish to work.
The Toolchain package contains development tools such as the compilers, linkers, debuggers, bmake, and other tools that are
used by the scripts that invoke the application build process. The Backing Sandbox package contains the actual libraries, APIs,
and scripts with which to build the applications. It also contains a growing number of sample applications that demonstrate the
API usage.
Various scripts from the backing sandbox greatly assist with the steps of development. For example, in the first required step, a
script is used to create a private signing key and certificate request. Once the partner is issued a certificate, the partner developer
can begin creating development sandboxes. This creation is again automated with a script. These sandboxes are directories in
which the developer creates applications. Sandboxes have a skeleton of useful subdirectories and build scripts, but they can
optionally be created to contain one or more sample applications as well.
Inside a development sandbox are two key top-level scripts. One of these invokes a build to compile the code into binaries, and
the other invokes packaging. The packaging can build multiple packages where the developer defines what goes into each one.
All of the generated packages automatically include the certificate and a signature. Packages can also contain other packages to
create easily deployable bundles. They are placed in one subdirectory of the sandbox and can be copied to the device for
installation. You can debug applications in their sandbox and on the device. In the VBE one can use the well-known GNU
debugger (gdb) to do post-mortem debugging with a core file or to attach to a live instance of gdbserver on the device.
Copyright ⓒ 2011, Juniper Networks, Inc. 13
Creating Innovative Embedded Applications in the Network with the Junos SDK
Application Examples
In this section we describe two real sample applications designed and implemented with the RE and Services SDKs. We start with
brief descriptions of the applications at a high level, focusing on important practical topics such as the useful software libraries
and data traffic steering. In turn we explain the creation of MoniTube, an IPTV monitoring and mirroring application; and
Equilibrium, a load-balancing and traffic classification application.
MoniTube
MoniTube is an application that can be used to monitor the quality of IPTV streams at any point where it is deployed on the
network. It can report several basic quality-indicating metrics. MoniTube can also mirror IPTV streams to other locations.
When MoniTube is installed, the Junos OS user interface is seamlessly extended to allow for its configuration and for reporting of
the video streams’ quality through operational commands. Configurations can then be entered to separately identify the streams
of interest for monitoring and mirroring. A MoniTube management application runs on the RE to load this configuration and
transfer it to a MoniTube service application running on a service engine.
The MoniTube application needs to see all packets of the streams it is to monitor and mirror, with no need to alter the original
streams. Therefore, the application’s packet processing functionality runs on a service engine that does not receive the original
streams, but rather copies of all its packets for monitoring.
To receive copies of packets, we take advantage of the Junos OS sampling feature, which enables an operator to send sampled
packets to a service engine, where service applications can process them. Our application needs to receive all packets selected for
sampling, so the sampling rate is configured as 1, indicating that every packet steered to the sampling framework is sampled to
the target service engine, and hence to the MoniTube service.
We use programmatically controlled firewall filters to select packets for which sampling is required. These filters narrow the traffic
selected for servicing. For example, if a variety of traffic flowed into the device through a given interface, a filter could be set up
to match all UDP traffic in 226.16.64.0/18 and 228.1.2.3/32 and select it to be sampled, and thus directed to MoniTube. With
programmatic control over the filters, the selection of traffic for sampling could be more dynamic if needed. Our simple approach
gives the operator the flexibility to select all or a subset of the device’s interfaces and traffic to be under MoniTube’s
management.
The MoniTube service is implemented as a multithreaded daemon running on the service engine. Thus, it follows the Services
SDK’s process model, but a plug-in model implementation of the MoniTube application is present in the Junos SDK sample
applications for comparison. In the process-model edition, the daemon’s main thread (control component) is primarily responsible
for communication with the management component; but more interesting are its many other real-time data threads dedicated to
packet processing. These threads poll the input packet queues, dequeue packets, perform the monitoring calculations, do any
packet manipulation for mirroring (e.g. changing the destination address), and then if the packet is not mirrored, dispose of the
packet, since it is a duplicate of the real selected traffic. If the packet is to be mirrored, we send it as modified during the
processing, enqueuing it into an output queue. This application architecture can easily be applied to dealing with real packets
instead of copies, where the original monitored packet is sent by default and not dropped. However, with sampled traffic, we
know we will never adversely impact the original IPTV streams.
Equilibrium
Equilibrium is an application that provides two simple functionalities that are implemented and potentially deployed as separate
services: load balancing and traffic classification.
The load balancing service has some configured facade addresses, and for each has a pool of real addresses. When it sees traffic
destined to a facade address, the service redirects it to an address from the pool with the least load, based on the number of
connections. The classification service provides the ability to redirect traffic to a single different destination address, but based on
a match with a destination port number rather than a destination (facade) address.
Once installed, just as with MoniTube, a management component runs on the RE, reading in any configuration for each of these
services. It passes this information down to the service’s control component running on the service engine. Multiple service
engines may also be configured to run one or both of the Equilibrium services, which work through service sets. The
management component is also responsible for gathering this information and for sending only the required policies to the
Equilibrium service running on each module.
14 Copyright ⓒ 2011, Juniper Networks, Inc.
Creating Innovative Embedded Applications in the Network with the Junos SDK
2000378-001-EN Feb 2011
Copyright 2011 Juniper Networks, Inc. All rights reserved. Juniper Networks, the Juniper Networks logo, Junos,
NetScreen, and ScreenOS are registered trademarks of Juniper Networks, Inc. in the United States and other
countries. All other trademarks, service marks, registered marks, or registered service marks are the property of
their respective owners. Juniper Networks assumes no responsibility for any inaccuracies in this document. Juniper
Networks reserves the right to change, modify, transfer, or otherwise revise this publication without notice.
EMEA Headquarters
Juniper Networks Ireland
Airside Business Park
Swords, County Dublin, Ireland
Phone: 35.31.8903.600
EMEA Sales: 00800.4586.4737
Fax: 35.31.8903.601
APAC Headquarters
Juniper Networks (Hong Kong)
26/F, Cityplaza One
1111 King’s Road
Taikoo Shing, Hong Kong
Phone: 852.2332.3636
Fax: 852.2574.7803
Corporate and Sales Headquarters
Juniper Networks, Inc.
1194 North Mathilda Avenue
Sunnyvale, CA 94089 USA
Phone: 888.JUNIPER (888.586.4737)
or 408.745.2000
Fax: 408.745.2100
www.juniper.net
Printed on recycled paper
To purchase Juniper Networks solutions,
please contact your Juniper Networks
representative at 1-866-298-6428 or
authorized reseller.
The motivation for using the Junos OS service sets was twofold. First, unlike the case of MoniTube, both services need to act
on real traffic transiting the device. A service set specifies an ordered set of services with this requirement, and one or more
sets may be associated with a service engine. Data traffic is redirected to the service set when filtering on interfaces detects
matched packets that need to be serviced. Second, we want to allow Equilibrium services to be run together on one service
engine, and potentially with other Junos or third-party services. The plug-in model achieves this aim: in general, plug-ins are
meant to work with service sets.
In accordance with the configured order in the service set, packets pass through the data event handlers of the Equilibrium
services and any other services configured in the same set. When seeing the first packet of a session, we look into our
configuration policies for facade address or port matches and store the corresponding action in the session context. For
subsequent packets of the same session, this action is available in the context and is immediately applied. This method
expedites servicing, as we simply retrieve the original action taken on packets of each flow.
Both MoniTube and Equilibrium are sample applications provided with the Junos SDK, where all code, build, and deployment
instructions are provided.
Conclusion
The Junos SDK enables applications to be embedded in the network by running them in the devices themselves on top of the
Junos OS. Juniper Networks uses the Junos SDK extensively to build its own software components and service applications,
and many customers, partners, and research organizations are already doing the same.
The Junos SDK enables network element programmability, but Juniper Networks views the Junos OS as a platform, not
just for such programmability, but as a host to innovative applications that allow differentiation to create value and deliver
tomorrow’s solutions.
About Juniper Networks
Juniper Networks is in the business of network innovation. From devices to data centers, from consumers to cloud providers,
Juniper Networks delivers the software, silicon and systems that transform the experience and economics of networking.
The company serves customers and partners worldwide. Additional information can be found at www.juniper.net.
Disclaimer
This work is an excerpt from the forthcoming book “Network Embedded Management and Applications” expected to be
published in 2011 by Springer Press.