INTELLIGENT Sciences, Chennai, Tamil Nadu, India E-mail:

INTELLIGENT
AVIONICS SYSTEM ONBOARD AN AIRCRAFT ENHANCING COMMUNICATION

N Sharma

Student, School
of Aeronautical Sciences, Hindustan Institute ofTechnology and Sciences, Chennai,
Tamil Nadu, India

E-mail: [email protected]

Abstract. Communications connect the flight deck to the ground and the
flight deck to the passengers. On?board communications are provided by
public-address systems and aircraft intercoms. Inefficient communication may
lead to a catastrophe risking lives and resources.  SKYVOIX, an intelligent system on board that
enhances safety and improves the communication of the aircraft by reducing
human error and increasing the awareness. The system is intended to be
centralized and installed in addition to the current systems. With high air
traffic and busy airspace, the transmissions are often misheard causing
communication failure which is undesirable. A backup for enabling communication
with the ground in emergencies is required which ensures active transmissions
and is intelligent enough to handle the gravity of situation. The passengers
who are often unaware about the routine and emergency protocols, busy with
their electronics, listening to music cutting them off with the environment
shall be benefited. The system works on encrypted conversions in real time
converting ATC, cabin crew’s instructions and delivering apt information to
appropriate sections which is achieved by Deep learning framework TensorFlow. The
system shall also be used for real time flight planning by ATC based upon
traffic density. The stability of the system is studied with effective layouts
reducing the errors and maintaining the overall neutrality. The on-board data
shall be stored, analyzedwhich shall help in the maintenance and crash
investigations. The system proves to be efficient, reliable and practical
enhancing the aviation safety and communication that can never be compromised
in aviation.

1. Introduction

With advancements
in aviation technologies the skies have become a reliable channel for
transportation be it for commercial sector, private or public. Safety has
always an important factor which plays a major role in aviation and has no room
for error. The industry is expanding with 3.7% annual Compound Average Growth Rate forecasting
passenger demand to double in next 20 years which refers to increase in air
traffic demanding more resources like aircrafts, man power, ground facilities
and most importantly the airspace which is difficult to expand. The airspace
can either be expanded 1. In height, this shall involve the development of new
technologies for engines and airframes considering the economics that may take
many years or, the other option 2. Along the span, which is hindered by
the fact of earth being 71% of water over which the controlling the traffic is
difficult and the policies of the air carriers and regional airspace. The only
possible solution is the evolution of better technologies that could help
aviation progress by handling future parameters with high efficiencies and standards. The aircraft should possess advanced sensing, computing
and communications, networking capabilities as well as onboard systems
integration and software modules. It is envisioned to robustly participate as
an intelligent node in a global network of air, satellite, ground systems,
ensuring quality information correctly, and reliably reaches the right place at
the right time for processing and decision making.

This work
emphasizes on the ways to increase safety standards by improvement in the
avionics. Intelligent avionics are such systems that can process their
computations making the interface versatile, accurate and producing data which
can further be processed explicitly. 

2. Claims

The
objective of the work is to establish a secure communication system that
processes on the ATC communications, in-flight communication, and flight data
in real time using Deep learning, Big Data analytics, Error correction, Neural
networks. The system is conceptualized thatshall address the following:
                       1.Smart IFE (In-Flight Entertainment) and
notification systems

                       2. ATC radio
communication systems

                       3. Flight data
Synthesis 

3. Description: 

        3.1
SKYVOIX IFE system and Flight Data Synthesis                                              

Some aircrafts are equipped with IFE systems and some are not in
order to save the costs. The system is important not only for entertainment but
majorly for cabin announcements and instructions. SKYVOIX system shall
help in creating awareness during the important phases of flight and
emergencies and also provide entertainment feed. Safety always comes first and
nowadays passengers prefer to isolate themselves from the environment by using
earphones plugged to their devices, poses major risk which has not been
addressed yet. The system shall convert the cabin announcements to text in
passenger’s preferred language and display on their Wi-Fienabled device. If the
aircraft or the device is not Wi-Fi enabled, a user based application is
designed that is discussed later. This system shall help in cutting costs by
residing over the current IFE systems which are being replaced by Wi-Fi IFE. The
system helps in removing the passenger units, SEB’s and wiring. 

            3.1.1 System specifications

1. Diverse architecture
          

2. Deep learning based real time audio to text converter of
multilingual capabilities 

3. Compatible with satellite linkage and current IFE systems

4. Converter can be used irrespective of aircraft being Wi-Fi
enabled

5. Dedicated user’s end application

3.1.2 Physical
architecture

The
architecture of the system is simple and separate. It is directly fed to the
raw analogous audio data from different sources. The design is such that the
total stability of the circuit is maintained and data is securely processed and
propagated. The schematic diagram of the system is synthesized to describe the
processes and hardware required. For the data to be accurate and easy to
process is encrypted and converted to digital format using the A/D converter
and then is processed through the error control algorithm into the converter
after which it is post processed and fine tuned 
through the D/A converter for obtaining readable format, then is
compressed and stored. All these processes take place in a single unit. This
converter unit is connected to the Display systems and the Wi-Fi system for the
output. The unit can be modified and programmed according to the need of  

                                                              
Figure 1SKYVOIX IFE SYSTEM

the
air carrier for instance updating of the entertainment data base like movies,
songs. With this architecture all the announcements and cockpit
recordings can be stored and locally and on the connected devices which can be
transmitted in real time.  This method is
used for Flight Data Synthesis.

3.2 ATC Communication System     

With
increasing air traffic, the airspaces are becoming congested and causing slow movement
of traffic. The ATC communication over radio being transmitted in heavy traffic
is difficult for the pilots as well as the controllers to follow. For this
problem the solution of text to audio conversion is implemented. This section
is in CROSS-REFERENCE TO the application US 11/934,607 of James J. Foskett which is
about the method of implementation but in contrary SKYVOIX is modified system
that processes the transmission and refines it using error control codes such
that no data is lost and uses deep neural networks for audio to text
conversion. The data is sequenced securely avoiding the mixing and can be
reviewed in real time with an informative system logging and displaying
airplane’s response.  These changes are defined for accurate, fast and secure
information propagation. The data transmitted between both the terminals are
stored and can be processed for analysis for preparation of accurate flight
paths in real time according to traffic density. This type of data comes under
BIG DATA and can be processed and analyzed using frameworks like Hadoop, Spark
or the Deep learning frameworks like Theano, Keras and TensorFlow.

3.2.1 Physical
architecture

In this system the same SKYVOIX IFE and audio to text converter
is used. For the system to establish the communication, architecture is defined
at both the terminals (The ATC and the aircraft).

         

                         Figure 2 SKYVOIX ATC COMMUNICATION SYSTEM

The architecture is mapped and shown in the form of a figure.
The additional hardware required would be the storage facility at the ATC end
and the information handling unit as well processing units at the both ends.

          

                      
Figure 3 INFORMATION HANDLING AND
RESPONSE

The architecture of
the Information handling and response is mapped in the figure. This system is
compatible to be used with CPDLC and resides over its limitations. Like the
controller settings, the settings can be changed in the aircraft as well. With
the unit of data processing and analysis the power is given to the computers to
alter and provide best flight paths to reduce congestion of the air traffic in
real time. The communication channel between both can be even radio or
satellite. For the radio type the data has more chances of error so requires
power and thorough refining. All the transmissions are stored securely for a
limited period and can be even monitored in real time.

 

3.3 Materials

Materials required
would be used for two functions one for the storage and the other for the
protections such that there is no hindrance in the communication process. The
materials with this capabilities are the Graphene based Nanocomposites that
most importantly have the ability to store the data howsoever the quantity of
storage may be limited that is why the data has to be relayed with the
technique already discussed

3.4 User’s End

The passengers
would automatically get notified when announcement will take place. If Wi-Fi is
available with or without internet the system will work if Wi-Fi is not
available then using an application the conversion will be done by accessing
the microphone of the passenger’s device. 
The users would mainly be the cabin crew and passengers for whom an
application is designed such that it is integrated with TensorFlow. The mobile
application being designed is for the Android platform using Android Studio
integrated with TF Speech. If internet connection is not established in the
aircraft in that case the IFE system described in previous section shall
kick-in providing the conversions over the Wi-Fi network. As shown in figure
the app can be directly used by cabin attendants for making announcements by
just texting into the network.  

 

Figure
4 SKYVOIX USER INTERFACE

 

4.Method

4.1 Deep Learning 

 Deep
learning is a branch to artificial intelligence. Deep Learning algorithms have great potential for
research into the automated extraction of complex data representations. Deep
Learning algorithms can develop a layered, and hierarchical architecture of
learning and representing data. Deep Learning in Big Data Analytics has become
a high-focus of data science. A key benefit of
Deep Learning is Big Data analysis that it can learn from massive amounts of
unsupervised data. This makes it a valuable tool for Big Data Analytics where
huge amounts of raw data are uncategorized. The
internal working of the system is achieved by implementing the concept of deep
learning which is closely related to machine learning and works upon neural
network algorithms. The neural networks can be trained to use encryptions such
that no agent interferes into the network and can even limit the passenger’s
internet surfing capabilities. The open source software library used for implementing
this technique is TensorFlow that is used for practicing machine learning and
evaluating large data sets.

1)    
Audio to text conversions

2)    
Real time mapping

3)    
Big Data Analytics:

 

·       
ATC Data

·       
On-Board Sensor and Recorder’s Data

 

A.   
Audio to text conversion

Deep
learning for speech recognition is used with 5 CNN’s. it is also implemented
with Tensorflow libraries. The training data is needed to be in batches with
pilot and cabin commands in different emotions and are needed to be fed in to
the neural network with learning rate as 0.0001 and weights are to be decided
appropriately. All segments shall be classified independently. IT is trained on
the last layer of the Google’s Inception model.

                                                  
Figure 5 Inception Model Architecture 1

Jianwei Niu in 11
used various features in their recognition system and combined DNN with HMM
reporting 92.3% accuracy on 6 classes of 7676 spoken Mandarin Chinese sentences.
H.M. Fayek in 12 explored various DNN architectures and reported accuracy
around 60% on two different databases eNTERFACE 13 and SAVEE 14 with 6 and
7 classes respectively.

 Sadly, we are not aware of any paper that used
Deep Learning for Speech Emotion Recognition on Aircraft transmissions within
and around.

 

B.    
Real Time Mapping and Big Data Analytics

Real time mapping is the automated flight planning of a
localized airspace depending upon the traffic statistics achieved by feeding
the airplane’s flight parameters like airspeed, heading, altitude, destination,
approach and runway allotted into the deep learning powered processing unit. The
flight parameters can be easily fed from the radar data. As this data is huge
in quantity and is being updated every second that is why it falls under the
Big Data and thus its analysis in order to meet the required output is to guide
the aircrafts safely abiding the airspace regulations. In this analysis Time
Series Prediction is used. Big data is also used and processed for maintenance
check which is determined by analyzing the sensor data in real time. This
method is already used by commercial giant Etihad Airways to improve its
economics and ergonomics. But this technique requires huge capital as big
satellite bandwidth is required. Instead going by the satellite transmission
altogether, the other method proposed is stored locally in the aircraft and
relayed within wireless network between hub and devices and when required is
down linked to ground or else near traffic such that a back up is created and accessed.

4.2 Error Control Codes

 The
error controlling coding is to be implemented to eliminate the redundancies in
the communication channel and the converter so as to increase the accuracy.
These codes implemented in deep learning framework are iterated many times to
refine the result and in the end providing a desirable result. It works as
Input data, pre-processing, neural networks, post process, output data. In the
mentioned applications sometimes it happens that the data suffers from
deficiencies which should be remedied before the data is used for network
training.Convolution codes work on symbol
streams of arbitrary length which are most often soft decoded with the Viterbi
algorithm that comes under FEC Algorithms. When
considering Hamming Code, the probability P (E) that the receiver commits an
error is given by:

P(E) = Pe/ ( Pc + Pe)

Where  Pc= (1 – p)n

For
(n.k) linear code:

Pe<= 2-(n-k) 1 – (1-p)n    4. Results The converter based on Deep learning convolutional neural networks was implemented using TensorFlow. The trained model shall be incorporated with the mobile application SKYVOIX using TF Speech. The training is done on the last layer out of the 5 convolutional neural layers using the Inception model. The training shall be implemented using real training samples of the flight announcements following sampling processes. With this information training data set is generated and then it is trained at 15000 iterations and appropriate learning rate set at 0.001. The training is done in batches which mean that the input of samples to fit the model is defined. The discussed method is the ideal process which shall be implemented on a powerful machine with GPU Boosting. Instead the process was implemented at a small scale with the retraining process as shown with a training set of photographs classifying aircrafts which shall be implemented in ATC tracking in video feed.        Figure 6 Re-Training The ATC traffic planning is done on the same deep learning platform, TensorFlow implementing Convolutional Neural Networks on the same Inception model used in the earlier case. The process for this would be the same but the training data would be different and would incorporate the data coming in from the radar namely present coordinates, altitude and magnitude The physical layouts as discussed earlier are compatible with the current IFE Systems like that of Thales IFE. As the layout has independent circuitry it does not hinder in the present system. It just draws the input from the sources that can be done by adding an extra node maintaining the transfer function. The power requirement should be 28VDC that is a standard for the aircraft instruments. 5. Conclusion and Discussions The question of how to improve the avionics for the modern and busy skies has been a popular research topic. This study proposed an effective method to improve the current technology with the help of data sciences which makes the proposed system intelligent such that the processes are automated and accurate that lays down the framework for increasing the safety in skies. Deep Learning is used for the same which is a sub-division of artificial intelligence. The implementation of Big Data and Deep Learning goes hand in hand. Its application includes Remote Sensing Data from satellites and being used for Image and object based classification which can also be implemented using TensorFlow API. Compared to traditional classification methods, the proposed model shall be able to do speech recognition in addition to voice recognition and converts it. The Deep Neural Networks makes processing becomes fast, in-depth replicating the functions of human brain. The module proposed for passengers is as important as communicating to ATC. The main aim for aviation is to improve transferability without compromising safety but today's scenario is the complete faith of passengers in the technology being totally ignorant. The device application should be able to monitor over that aspect making a richer user interface. The concept of Big Data and deep learning is to be applied for the aircraft monitoring as well data synthesis with minimum data loss and better compressibility which shall determine the feasibility of the system. A significant move to intelligent systems may make the sky easier for travelling in days to come and even maintain considerable low load factor on the traffic controller and pilots reducing the Human Error.