Dealing With Python Apps The punch story with python is an old one. We implemented an elasticsearch aggregator tool in the Brad release (now deprecated), we also leveraged the elasticsearch curator application that we provide as a deployable administrative service in the Craig release. More importantly, we decided to fully support pyspark and to make Read more about Packaging Python[…]
Craig Release Latest News This 2019 summer was quite active, and I am happy to share lots of interesting news regarding the current and future punch releases. Before We Start Maybe you are not familiar yet with the punch. Here is a quick recap. The punch is a ready-to-go-production data analytics solution. It is built Read more about Craig Release[…]
Big Data Paris 2019 Workshop Tuesday 12th of March, 14H00 Machine Learning for Critical Systems Damien Fontanes, Dimitri Tombroff Abstract In real-life critical systems and applications, accessing the data to test, tune and run artificial intelligence algorithms involved several under-estimated problems. The first challenge is mainly to provide consistent tools and execution engines usable for Read more about The Punch at Big Data Paris 2019[…]
The Craig punchplatform version 5.1.2 is released. This post gives an overview of the new features. New Features The Craig punchplatform 5.1.2 has a number of important new features. First off, it leverages many important component updates: Elastic 6.5.4 stack Kibana 6.5.4 Spark 2.4 Siddhi Rules 4.3.17 It also brings in a number of new Read more about Craig Release 5.1.2 Annoucement[…]
In the punch, we do not use logstash for high-performance logs parsing. Why is that? Mainly because logstash is nor easily scalable nor does it provide an end-to-end acknowledgment pattern. This is a serious lack because we cannot afford to lose logs whatever happens. We thus selected an alternative technology (apache storm) to run our equivalent input/filter/output processors, Read more about News From Logstash[…]
Punch Data Science Meetup When: on the 15th of October. Where: in Rennes at the Google atelier numérique. Organised as part of the Rennes Machine Learning Meetup /French Tech Speaker: Simon Grah from Thales Theresis Research Lab Topic: Maritime Traffic Anomaly Detection We are very happy to have Simon animate our next meetup in Rennes. Many thanks to Camille Saumard (from lumenai), to invite Read more about Punch Data Science Meetup[…]
Description If logs sent using tcp contain extra carriage returns, the input topologies reading drop to a low rate. The reason is that these carriage returns are silently dropped (which is correct) in a way to cause Storm engine to think it should wait before getting the next logs. This (storm) behaviour is configured using Read more about (PP-1703) additional carriage return make input topologies very slow[…]
Description Kafka brokers suffer from a memory leak (https://github.com/apache/kafka/pull/4307) related to the metric reporting feature. This is unfortunately not a feature that can be disabled. This leak affects version 0.10.x and 0.11.x releases and has been fixed in release 1.x only. Impact This leak will slowly fill the heap of kafka broker processes. It is easily identifiable Read more about (PP-1841) Kafka memory leak[…]
We have 5 git repositories, our documentation is automatically generated. We use only maven and git to build our complete packaging. Nothing tremendous. In order to build and keep track of things, here is what we do. Simply said, we like it.
The story of the punchplatform is worth having a look at. The punch team started small, is still very small, yet designed a platform that has now a great positioning because of the strength of its internal components (ElasticSearch, Kafka, Storm and Spark), and its unique value added. Part of that value is technical (there Read more about A Brief Punch History[…]