We are seeing a rapidly changing world when it comes to the energy sector. Day by day the energy needs are on the rise and hence, the choice of renewable energy becomes ideal, as the cost is low and implementation is quick. Solar energy leads in market share when it comes to renewable energy. Hence it becomes very crucial to track and understand the power generation and distribution.

We at Mobiuso have been working with partners in alternative energy sector to provide software application & analytics services which pull data from solar devices & provides an effective way of tracking, monitoring & analyzing that for additional insights. We have implemented a complete end-to-end systems which includes the software component & the deployment & operating environment over cloud.

The Front-end

At present, the front-end exists as a responsive web application, powered by AngularJS. On the backend, we use NodeJs & MongoDB.

Middle Layer

We created a data acquisition process, that basically works as a middleman between the solar device and the a third party service. The acquisition process continuously listens to the solar devices for incoming data. It uses a TCP protocol to efficiently handle the data. Once it gets the data, the acquisition process will forward it to another process called RabbitMQ, a message broker. Further on, the RabbitMQ will be responsible for queuing the data to the aggregation server. The aggregation server will pull the data and process it. The data processing in the aggregation server is based on crucial algorithms with inputs like energy, power, voltage etc from the solar devices.

The aggregations server will also send acknowledgement to the solar devices through the RabbitMQ. Further on the processed data will be stored in a database managed by MongoDB.

There is another web server that works independently from system, but dependent on the MongoDB database. The main job of this server is to provides real time actionable intelligence with custom reports and alerts for top management, project managers and O&M team.

Ensuring that the system works as expected

The challenge in the testing comes from the future addition of solar devices to this system, and this system should not fail.

We have created a process to send data to the server which takes different parameters such as, No. of devices, Frequency in Seconds, Max packets per devices.

By using  this simulator, we did the testing with different RAM and CPU Cores. by using this we found how many devices could be managed with the current system.

We initially designed the system for processing almost 5000 – 10000 solar devices. The complexity involved in this itself was huge as the data will coming in parallel at the rate of 10 packets per minute, where each packet holds 100 bytes. Hence the approximate handling power ranges between 36 * 10^8 and 72 * 10^8 bytes per day. However later on we had to scale up the capability to process almost 1 lakh devices per day. This transition itself was a huge challenge for us. We implemented a cluster to handle the load and divide it among various cpu core.

In future we can add more cpu core to handle the load and thereby increasing the number of solar devices the system could handle.

The solar device always sends the date and time  in UTC format. However, the Aggregation server, that process the data, converts it into the local timezone and stores the result in the database.

Furthermore, we have implemented control features in the user applications, such that, we could dispatch commands to do factory reset, add device details and other configuration parameters for the solar devices.

Now the data stored in the MongoDB, will be plotted to charts, graphs at the client end (mobile, PC). We have already used HTML5 JavaScript Charts for displaying real time charts on client dashboard. This charting library is great for date and time visualization.

Ours is a real time data processing system, hence any errors or crashes that causes it to stop working will be detected immediately and will automatically restart all the system processes so that the system comes back online. Furthermore, it also sends email to the system admin regarding the system failure with adequate details. We currently use the MONIT utility provided by the Linux OS to detect and monitor the our system.

Next steps

The next phase of the evolution for the system would be to introduce powerful analytics & leverage Machine Learning to provide even greater insights & control. A mobile extension to existing web capabilities can also be on the cards. Please stay tuned for what we are doing next. It’s going to be one exciting story for sure.