It seems everyone in Big Data is doing Weekly / Bi-weekly / Monthly review. Here is my subscription list

• Hadoop Weekly by Joe Crobak (Weekly)
• This week in #Scala by CakeSolutions (Weekly)
• Log Compaction - Highlights in the Apache Kafka and Stream Processing Community by Confluent (Monthly)
• Databricks Bi-Weekly Apache Spark Digest by Databricks (Bi-Weekly)
• Lightbend Tech Digest by Lightbend (Monthly)
• Montly Review by dataArtisans (Monthly)

Now I'm writing up a review based on reviews.

Releases

A bunch of maintenace releases from the open source community.

• Apache Storm made a maintenance release 0.10.2 for 0.10.
• Apache Flink released 1.1.3, bugfix release for 1.1 series. RocksDB state backend users are recommended to use the "fully async" mode such that upgrade to 1.2 will be easier.
• Apache Spark also made a maintenance release 2.0.1 containing 300 stability and bug fixes.
• Apache Calcite released 1.10.0 and 1.9.0, and mainly improved Druid Adapter.

Conferences

• Flink Forward 2016 was held from Sep.12th to Sep.14th with 350 attendees and over 40 speakers. Videos and slides for all sessions are available now. After a quick glance through, I come up with this list of mainly technical talks which nicely capture the current status and next steps of Flink.

  • Connecting Apache Flink to the World: Reviewing the streaming connectors
  • Declarative stream processing with StreamSQL and CEP
  • Dynamic scaling: How Apache Flink adapts to changing workloads
  • Running Apache Flink everywhere: Standalone, Yarn, Mesos, Docker, Kubernetes, etc.
  • Scaling Stream Processing with Apache Flink to very large State
  • Streaming ML with Flink
  • Taking a look under the hood of Apache Flink’s relational APIs
  • The Future of Apache Flink

• Reactive Summit 2016, the first ever Reactive Summit, took place Oct.4 - Oct.5 in Austin. All talks are now available. I haven't gone through the topics but Implementing an akka-streams materializer for big data by our Kam is definitely worth checking out.

Articles

• 7-big-data-tools-to-ditch-in-2017. Both MapReduce and its streaming counterpart Storm make the list. Wait, are you sure to ditch Java ?

• Scalable Stream Processing: A Survey of Storm, Samza, Spark and Flink

  give an overview over the state of the art of stream processors for low-latency Big Data analytics and conduct a qualitative comparison of the most popular contenders

• 4-min read on How Kafka’s Storage Internals Work. A big fan of such short informative stories.

• Jay Kreps has a new masterpiece, Sharing is Caring: Multi-tenancy in Distributed Data Systems

  You see hundreds of blog posts on benchmarking infrastructure systems—showing millions of requests per second on vast clusters—but far fewer about the work of scaling a system to hundreds or thousands of engineers and use cases. It’s just a lot harder to quantify multi-tenancy than it is to quantify scalability.

  Can't agree more ! A multi-tenancy benchmark, anyone ? I'll leave you here to think more about this topic.