TodoBI - Business Intelligence, Big Data, ML y AI TodoBI - Business Intelligence, Big Data, ML y AI
Últimos
Novedades en Knime 5.3 10 consejos para monetizar los datos de tu empresa 17 KPIs para medir un proyecto de Data Governance 15 recursos para aprender gratis Microsoft Fabric Diccionario de IA Generativa Curso de Inteligencia Artificial para técnicos Windmill, alternativa a Apache Airflow

LinceBI, la mejor solución Big Data Analytics basada en Open Source

Curso Online de Power BI (17, 18 Febrero)

Pentaho Analytics

Material Big Data

Metodologias Agiles para Analytics (Business Intelligence, Big Data)

List of Open Source solutions for Smart Cities - Internet of Things projects

Admin may. 23, 2019 0
open sourceiotsmart citiesbig data

Increasingly projects are carried on so-called 'Smart Cities', supported by Big Data, Internet of Things... and the good news is that most of them are made with Open Source technologies. We can share, from TodoBI.com our insights about these technologies
Making a city “smart” involves a set of areas we will outline below: Without IOT (Internet Of Things), there will be no Smart City. 
Since automatic collected data is the most efficient way to get huge amounts of information, devices connected to the internet are an essential part of a Smart City.
The way we store and process data from city is generally using Big Data and Real Time Streaming technologies. 
The final goal where more innovative and custom analysis can be achieved using Artificial Intelligence and Machine Learning. Finally I would include Apps, as usually this kind of solutions is consumed in mobile devices. 

Here we outline the common process of building a Smart City solution: 
-Choose data 
-Connecting devices 
-Design Data Storage Infrastructure 
-Real Time Events and Notifications 
-Analytics -Visualization (Dashboards) 
  1) Choosing Data  
In a city there are three basic sources of data: citizens, systems, sensors. Use the available information of users, on social networks, informations systems, public statistical information offered by the administration. 
A typical example is user with geolocalization enabled in twitter. Information about the systems and services in a city are sometimes available in open data sources. An example could be the water or electricity consumption. 
Last but not least, sensors. A city hoping to become “Smart” has to intend to provide automatic information of its environment, and that could be achieved using sensors. Sensors can be anywhere
2) Connecting Devices
Devices (sensors) connects with the real time data streaming and the storage infrastructure using efficient communications protocols, that using light weight packaging and asynchronous communications.
Examples of some communications protocols used:
MQTT (Message Queuing Telemetry Transport) Websocket (bi-directional web communication and connection management)
STOMP (The Simple Text Oriented Messaging Protocol)
XMPP (Extensible Messaging and Presence Protocol)
3) Design Data Storage Infraestructure  
The Data Storage Infrastructure for a Smart City solutions has special characteristics, due to the diversity and dynamism of its sources. 
Time series DB are frequently used, because of the time evolution of data catched by sensors Some examples of this kind of DB are InfluxDB and Druid . 
Another DB commonly used in Smart Cities project are MongoDB (json format advantages), Cassandra (fast insertion advantages), Hadoop (big data frameworks advantages)
Some samples

4) Real Time events and notifications
Usually Smart Cities solutions have needs for real time notifications on events. To accomplish such requirements the system must have a Stream Analytic engine, that can react to events in real time and send notification. This characteristics bring us some technologies related to this; Storm, Spark Streaming, Flink, WebSocket, Socket.IO
IoT Frameworks:
● Node-RED


Node-RED is a tool for wiring together hardware devices, APIs and online services in new and interesting ways.
The light-weight runtime is built on Node.js, taking full advantage of its event-driven, non-blocking model. This makes it ideal to run at the edge of the network on low-cost hardware such as the Raspberry Pi as well as in the cloud.
The flows created in Node-RED are stored using JSON which can be easily imported and exported for sharing with others.
An online flow library allows you to share your best flows with the world

● PubNub


     PubNub is a Data Stream Network, that offers infrastructure as a service. With PubNub,  we can use the infrastructure provided and connect our devices, designing our architecture and simply get advantages of all this.
PubNub has 5 main tools:
-Publish Subscribe (Allows Real Time Notifications of Events to users)
-Stream Controller (Allows managing channels and groups of channels)
-Presence (Allows notifications when users login or leave the system, or similar behaviour, device availability for example)
-Access Manager (Allows administrators, to grant or deny permitson users of the systems)
-Storage & Playback (Provide storage for messages,and allows messages retrieval at later time)

● IoT-AWS


AWS IoT is a platform that enables you to connect devices to AWS Services and other devices, secure data and interactions, process and act upon device data, and enable applications to interact with devices even when they are offline

5) Analytics and Visualization


You can show real time dashboards, reports, OLAP Analysis using tools like Pentaho . See samples of Analytics   

Imagen del autor
Admin
Website