Brad Release 4.0.0

We are happy to announce the Brad punchPlatform 4.0.0 release.

New Features

The Brad release has many significant new features coming either from the many new features of the  Elastic 5.* and the Ceph luminous v12.2.0 release update, or from the new punchplatform features.

Elastic 5.x

Checkout the elastic 5.0 new features list.  Here is a selected list with a focus on end-users experience :

• Instant Aggregations
• Support for IPV6 type mapping
• Elasticsearch 5.0.0 will use Lucene 6.2, a major version upgrade of Lucene:
  • performance improvement
  • a new fast and efficient numeric format called points
    • Based on a balanced K-D tree https://en.wikipedia.org/wiki/K-D-B-tree
    • Used for all numeric data (geo, bigint, ranges, ipv6)
• New numeric types : half_float and scaled_float

Kibana 5.x

The Kibana 5 release introduces many important new features. Using Kibana 5 is a complete new experience. In particular :

• The visualisation and dashboard editors are much improved.
• Development console
• Integrated support for time serie visualisations (timelion)

We suggest you watch this video for a good introduction.

Shiva 4.0.x

The Shiva distributed scheduler supports many new important features that makes it production ready. It is now internally used to manage user services such as the pygragator aggregation service.

• advanced task monitoring
• full support for periodic task handling

Punchplatform Pygregator (4.0.x)

Pygregator is the aggregation service to let you easily precompute aggregations on too high cardinality data.  In Brad, pygregator

• supports arbitrary lucene queries
• is now integrated as a channel inner task, managed by shiva.

Punchplatform Data Injector (4.0.x)

The platform injector has a number of new features :

• full support for server side mode : makes it extra easy to bench or test end to end processing.
• full support of lumberjack client and server mode
• elasticsearch destination : inject your generated documents directly to Elasticsearch.
• support full json configuration format, even for the payload specification.

Punch and Parsers

• The punch language now supports the dissect operator, an efficient and optimized alternative to the grok well known operator. Refer to this blog ?
• A new punch sublime text editor is available, as well as a online punch debugger. Please refer to the documentation.
• Parser design has ben improved in particular to transport all the header informations on the side, letting you write parsers only on the relevant part of your data. This in turn makes the standard log parsers more modular and efficient.

Batch Search Service

The punchplatform batch service let users design, configure and submit extraction or aggregation queries  whose results can be retrieved later as CSV files or directly visualised  Kibana.

This feature compensates for the lack of  a Kibana export function for large amount of matching records (e.g. 1.000.000), and provides user with a monitoring service for managing asynchronous long-running searches.

Refer to the punchplatform documentation.

Breaking Changes

This section discusses the changes that you need to be aware of when migrating your application to brad punchplatform-4.x  releases.

The central change is the update from Kibana 4.x to Kibana 5.x version. Refer to elastic kibana breaking changes page. You will have to migrate your dashboards.

Next comes the upgrade from Elasticsearch 2.x (Avishai)  to Elasticsearch 5.6. Watchout, this will require a full cluster restart. Refer to elasticsearch breaking change pages (5.0, 5.1,  5.6). There are a number of important changes that directly or indirectly.

To design a complete and safe upgrade procedure, contact us at contact@punchplatform.com.