Platform architecture, design and sizing oriented subject.
Overview Apache Nifi is a popular technology to handle dataflow between systems. Using a simple interface to create diagrams, you can manage where the data goes and how is it processed. Just like many similar frameworks, Nifi provides a bunch of built-in nodes to deal with various data sources or Read more…
Dealing With Python Apps The punch story with python is an old one. We implemented an elasticsearch aggregator tool in the Brad release (now deprecated), we also leveraged the elasticsearch curator application that we provide as a deployable administrative service in the Craig release. More importantly, we decided to fully Read more…
In the end of 2016, we integrated Ceph in Punchplatform. This article gives a feedback and explains what are the major advantages of using a Ceph cluster on a Punchplatform instead of standard storage solutions. User features Aside from reducing storage costs through well-known erasure-coding, Ceph in Punchplatform offers some Read more…
The punch is deployed in production for small, medium or large-scale applications, on the cloud or on premise, for various use cases such as cybersecurity, applicative or system monitoring. In this short blog we highlight a few concrete sizing example to give you a flavor of the hardware and server Read more…
Context While working on a surveillance solution an interesting problem arose: an analysis component had to be built which had to be light, robust and highly adaptative. This component consists of a set of sequential or parallel tasks, each providing a unitary treatment. The solution is built on top of Read more…
The new elastic 5 GA is out. It provides many improvements and great new features. In this post we have a look at the new Ingest nodes, that provide a way to deploy logstash filters without the burden of deploying and running logstash processes. Each running an input-filter-output pipeline. Since Read more…
The punchplatform is deployed in a number of industrial projects. By now you hopefully know the punchplatform is built extensively on top of Elasticsearch. We run production for several years now, and some of our elasticsearch clusters are fairly big and hold up to a year of data, parsed and Read more…
Among many other useful things, big data platforms lets you design data pipeline, fetching or receiving data from one side, applying some transformations, then pushing the resulting data further. If you run a punchplatform, you are likely to do that to collect then parse your data into Elasticsearch, in turn Read more…