Agenda 2020

February 27, 2020

11.10 – 15.30 Simultaneous sessions

16.10 – 17.35 Roundtables sessions

Parallel roundtables discussions are the part of the conference that engage all participants. It has few purposes. First of all, participants have the opportunity to exchange their opinions and experiences about specific issue that is important to that group. Secondly, participants can meet and talk with the leader/host of the roundtable discussion – they are selected professionals with a vast knowledge and experience.

There will be 2 rounds of discussion, hence every conference participants can take part in 2 discussions

16.15 – 16.55    1^st round

17.00 – 17.40   2^nd round

1. Managing a Big Data project – how to make it all work well?

Data Scientists, Data Engineers, DevOps Specialists, Business Stakeholders – all these people come from different worlds, but need to work close together in order to make a Big Data project a success.
Let’s discuss about our achievements as well as… spectacular failures when it comes to communication, cooperation and meeting ones expectations.
We’ll be talking about methodologies, tools, best practices and, so called, human element.

2. Analytics and Customer Experience Management on top of Big Data

How to ensure successful adoption of Big Data and Analytics systems? It is a challenge for most organizations. Let’s discuss how to promote user-centric approach, leverage experience design and manage user expectation on Big Data projects. I would be happy to hear you opinion and answer your questions, based on my practical experience applying Design Thinking and architecture design methodologies. I believe this conversation will be interesting for Architects, Tech Leaders, Product Managers and C-level folks.

3. Data visualization, how to visualize large, complex and dirty data and what tools to use

Data visualisation is great tool at explaining data – best we know right now. But our data volume grows every day and we often hit hard limits of current visualisation systems. How we can approach this problem so we still can analyse and explore data? From interactive interfaces that link many visualisations to machine learning algorithms that pick best chart type and parameters. It is really interdisciplinary issue, so let’s share our knowledge!

4. Practical application of AI

Industry 4.0 and AI – are we ready for the 4th industrial transformation? Who should be the beneficiary of the Industry 4.0? Key barriers to the implementation of AI projects in organizations. Real cases of AI in Industry.

5. The need for explainable AI

With the spread of AI-based solutions, more and more organizations would like to understand the reasons for system decisions. It’s especially interested in regulated industries. The session will cover so-called white-box methods, as well as modern approaches to AI explainability which allow understanding more complex models.

6. Real-life machine learning at scale using Kubernetes and Kubeflow

How to build a machine learning pipeline to process 1500 TB data daily in a fast and cost-effective way on Google Cloud Platform using Kubeflow? How to serve TensorFlow model with almost 1M requests per second and latency < 10ms on Kubernetes? Is Kubernetes and Kubeflow ready to serve data scientists?

7. Big Data on Kubernetes

Kubernetes found its place in microservices world. More and more teams are betting on Kubernetes as their goto platform for deploying business applications. What about Big Data? Can our ETLs also make the move? During the roundtable discussion we’ll discuss how Kubernetes can be utilised as a runtime for Big Data jobs. Is current tooling ready for being deployed in Kubernetes containers? What the potential shift means for used storage technologies? Finally, will Kubernetes democratize Big Data work and we’ll move from central data lakes to distributed data meshes?

8. Best tools for alerting and monitoring of the data platforms

Let’s discuss what should be monitored in data platforms? What are best tools for particular use cases? What is not recommended?

9. What to do with my HDP/CDH cluster with new Cloudera licensing model

After the merger with Hortonworks, Cloudera becomes a single vendor that builds a distribution that consists of major components from so called Hadoop Ecosystem (e.g. Hadoop, Spark, Hive, Ranger). While these components itself are open-source, access to binaries that are critical to install/upgrade the clusters will be limited to only customers who purchase a paid subscription. This means that thousands of the companies that currently use Hadoop for free, will need to decide what to do next. Should I pay for a subscription or compile own binaries to build own distribution? Should I stop using on-premise Hadoop and go to the public cloud instead? During this panel we explore this topic and try to answer these questions based on our vendor-neutral experience when working with our customers who have large production installations of HDP/CDH clusters.

10. Addressing challenges of modern analytics with Snowflake

Data is the new oil. More and more companies understand the value of data to optimise their core business or enter new business fields. They want to analyse data to enhance their internal processes, the way how they work with customers or how they collaborate with external parties such as suppliers, partners etc. All this is not trivial, requires the right skillset and an appropriate technology. The cloud promises scalability, elasticity and resources on demand. But a cloud native architecture is mandatory to leverage these features. Snowflake is built for the cloud, separates storage from compute and offers everything people expect from the cloud. Even a data marketplace for collaboration and monetisation is available that makes sharing and exchanging data globally easy. All this independent from a cloud vendor and capable for multi-cloud setups.

11. Being efficient data engineer. Tools, ecosystem, skills and ways of learning

What does it mean to be a productive (data) engineer? Is it about the tools we use? Is it the mindset we have? Is it the environment we are surrounded by? Let’s share and discuss war stories, learning resources, methodologies and libraries that help us escape the gumption traps in the daily life of an engineer. Discussion will be divided into 4 areas: debugging, implementation, communication and learning.

12. Data discovery – building trust around your data

Worldwide growth of data has changed business landscape forever. Multiple organizations undergo transformation triggered by the data revolution. While one can understand the benefits of collecting bigger data volumes, it has revealed additional challenges when trying to effectively use it. Ability to explore the data and increasing compliance demands force us to think about solutions to leverage power of metadata. Data description evolve from being a simple schema definition to catching application context, behavior and how it is changing over time.
Let’s discuss data discovery in context of use cases, technologies and possible challenges.

13. SQL on Big Data for batch, ad-hoc & streaming processing

Data Analysis is the key factor in data-driven decision culture and SQL is the omnipresent language for deriving information from data. Today, even small companies have huge data sets while huge organisations have enormous ones. With the advent of technologies aiming at replacing and unifying the ones we used previously, we only have more complex and more heterogeneous data landscapes. How to query the data to fuel key business decisions? How to handle data ingestion? When do we need it?

14. The Latest and Greatest of Apache Spark

Apache Spark is a fast and general engine for distributed in-memory computations on a massive scale. Spark 3 is in preview and expected to be released in the first quarter of 2020. What features are you waiting for and what problems do you hope to solve with Spark 3? The roundtable is to share and discuss problems we want to solve with the new features coming in Spark 3.

15. Serverless data warehousing – big data, the cloud way

Definition what is serverless warehouse. List of solutions which are considered serverless. Data ingestion. Data storage. Data processing, pricing and cost efficiency. Advantages and disadvantages of both serverless and on premise.

16. Stream processing engines – features, performance, comparison

Streaming systems are gaining more and more attention and we don’t expect this trend to slow down. Currently there are a few engines on the market. At this roundtable we will share our knowledge about similarities and differences among them in various areas. What are the strengths, weaknesses and constraints of each of them? Is there a niche for each of them or maybe a final winner should emerge?

 17. Snorkel Beambell – Real-time Weak Supervision on Apache Beam 

Deep Learning models have led to a massive growth of real-world machine learning that allows Machine Learning Practitioners to get the state-of-the-art score on benchmarks without any hand-engineered features. The challenge with continuous retraining is that one needs to maintain prior state (e.g., the learning functions in case of Weak Supervision or a pretrained model like BERT or Word2Vec for Transfer Learning) that is shared across multiple streams. Apache Beam’s Stateful Stream processing capabilities are a perfect match to include support for scalable Weak Supervision.

1. Managing a Big Data project – how to make it all work well?

Data Scientists, Data Engineers, DevOps Specialists, Business Stakeholders – all these people come from different worlds, but need to work close together in order to make a Big Data project a success.
Let’s discuss about our achievements as well as… spectacular failures when it comes to communication, cooperation and meeting ones expectations.
We’ll be talking about methodologies, tools, best practices and, so called, human element.

2. Bring Data as Products to consumers

How do you define data products? What mindset and approach we should have to make Product approach possible in Data? Let’s brainstorm about implementation of the approach, opportunities and values it brings.

3. Data visualization, how to visualize large, complex and dirty data and what tools to use

Data visualisation is great tool at explaining data – best we know right now. But our data volume grows every day and we often hit hard limits of current visualisation systems. How we can approach this problem so we still can analyse and explore data? From interactive interfaces that link many visualisations to machine learning algorithms that pick best chart type and parameters. It is really interdisciplinary issue, so let’s share our knowledge!

4. BI platfom: a choice or a circumstance?

We take a lot of effort to design datalakes, pipelines and data warehouses. But don’t forget what was the original aim at the very begining of this road: to get some insight. In order to utilize petabytes in orginzed and governed manner we need a solid BI Platform that suits our needs. A well chosen one is not only icing on the cake – it makes all data efforts reasonable and drives our business head-to-head with competition.
During this round table session we will discuss the importance of choosing the right BI platform, exchange best practices (and some bad ones too) and focus on currnet trends in the area of BI.

5. Real-life machine learning at scale using Kubernetes and Kubeflow

How to build a machine learning pipeline to process 1500 TB data daily in a fast and cost-effective way on Google Cloud Platform using Kubeflow? How to serve TensorFlow model with almost 1M requests per second and latency < 10ms on Kubernetes? Is Kubernetes and Kubeflow ready to serve data scientists?

6. Databases in Kubernetes: from bare metal to cloud native

Initial cloud-native conquests started with stateless services but gradually turned towards data management systems. DBMS have relied on centralized, bare metal servers for decades. Cloud-native architectures are a big new technology shift for such systems. Many commercial and open source databases already provide cloud-native adoptions of their products. Among the most interesting cloud-native converts are analytic databases, which imply additional requirements to storage and clustering techniques in order to run fast analytic queries over billions, or even trillions, of rows. Examples include the MySQL clustering project Vitess, which has recently reached CNCF graduation level, and ClickHouse, an extremely fast and scalable analytical database that is being converted to cloud-native operation by Altinity.

Join me to discuss various aspects of running databases in Kubernetes. This is a new technology that has a lot of caveats, such as storage. At the same time databases in Kubernetes promise substantial benefits to the users of such applications as well as companies that operate them. We will explore these issues as well as the path to maturity.

7. Scale Your Logs, Metrics, and Traces with the Elastic Stack From traditional applications to microservices and Kubernetes

How do you tackle your monitoring and observability problems? There is a high chance that you are using the Elastic or ELK Stack and this session is all about making it scale: From easier collection of data, to scalable multi-tier architectures, and the lifecycle of your data including the deletion.

8. From on-premise to the cloud: an end to end cloud migration journey

The goal of discussion is to share various experiences and ideas around tackling the migration challenge. Together we will try to nail down the benefits, obstacles and solutions that can help in our journey from on-prem to the cloud.
The plan is to identify the important areas of the migration and to share the lessons we’ve learned from those.

9. Challenges of building a modern & future-proof data processing platform

The speed of changes in IT and our companies seems to never stop increasing, especially in the field of BigData. To keep up with it we need to move fast and be smart about it, but how to actually achieve that? How to predict future needs for processing and tools? How to prepare for it? What kind of trade offs we can make? We’ll try to answer that together and share good practices and experience during this session.

10. Hadoop is dying, long life to HDFS, what are your options and plans for sustainable advanced analytics and machine learning?

Many enterprise organizations, particularly in financial services and telco, have built massive data lakes in Hadoop, specifically HDFS. However, Hadoop never lived up to its hype as a data warehouse replacement and languished as storage option in on-premises data centers. On top of that, the top two HDFS vendors – Hortonworks and Cloudera – have merged and the third, MapR, has been sold off to HPE in a fire sale.
What is your escape plan and alternatives for applying advanced analytics and machine learning to your growing data volumes? Will you adopt object storage as a cost-effective, data storage repository? Is the separation of compute and storage the future database architecture to manage variable workloads?
Join this roundtable discussion to get to the bottom of these top-of-mind questions and learn the emerging trends and options to build modern data pipelines for advanced analytics and Machine Learning over vast amounts of data?

11. Being efficient data engineer. Tools, ecosystem, skills and ways of learning

What does it mean to be a productive (data) engineer? Is it about the tools we use? Is it the mindset we have? Is it the environment we are surrounded by? Let’s share and discuss war stories, learning resources, methodologies and libraries that help us escape the gumption traps in the daily life of an engineer. Discussion will be divided into 4 areas: debugging, implementation, communication and learning.

12. Data discovery – building trust around your data

Worldwide growth of data has changed business landscape forever. Multiple organizations undergo transformation triggered by the data revolution. While one can understand the benefits of collecting bigger data volumes, it has revealed additional challenges when trying to effectively use it. Ability to explore the data and increasing compliance demands force us to think about solutions to leverage power of metadata. Data description evolve from being a simple schema definition to catching application context, behavior and how it is changing over time.
Let’s discuss data discovery in context of use cases, technologies and possible challenges.

13. SQL on Big Data for batch, ad-hoc & streaming processing

Data Analysis is the key factor in data-driven decision culture and SQL is the omnipresent language for deriving information from data. Today, even small companies have huge data sets while huge organisations have enormous ones. With the advent of technologies aiming at replacing and unifying the ones we used previously, we only have more complex and more heterogeneous data landscapes. How to query the data to fuel key business decisions? How to handle data ingestion? When do we need it?

14. The Latest and Greatest of Apache Spark

Apache Spark is a fast and general engine for distributed in-memory computations on a massive scale. Spark 3 is in preview and expected to be released in the first quarter of 2020. What features are you waiting for and what problems do you hope to solve with Spark 3? The roundtable is to share and discuss problems we want to solve with the new features coming in Spark 3.

15. Managing workflows at scale

How to build and maintain thousands of pipelines in the organisation? What are the biggest pain points in orchestrating hundreds of ETLs? What open source and managed solutions are available?

16. Stream processing engines – features, performance, comparison

Streaming systems are gaining more and more attention and we don’t expect this trend to slow down. Currently there are a few engines on the market. At this roundtable we will share our knowledge about similarities and differences among them in various areas. What are the strengths, weaknesses and constraints of each of them? Is there a niche for each of them or maybe a final winner should emerge?
Let’s share our experiences in using streaming engines as well as our predictions about their future.

17. Data Auditing

Have you ever encountered a situation when your pipeline (or system) produced less (or more) data than expected? Has it lost your data? Or maybe that data has never reached the source you read (or reached, but delayed too much)? Have you ever seen too many empty (or NULL) fields? Or a field called `age` with negative values? Or maybe you encrypted some data thoroughly enough, that no one could read (decrypt) that data anymore? Last, but not least, what to do if you detect a problem (or, do you even monitor your data? Do you have any alerts set?). Can you backfill, mutate or leave a wrong dataset? Let’s talk about different ways to ensure that data you produce makes sense.

Despite the spread of dedicated AI platforms, ready-to-use ML libraries and tons of data available, running successful large-scale AI/ML projects still faces technical and organizational challenges. According to some studies, 8 out of 10 such projects fail. This panel will explore the necessary technical prerequisites that a company should introduce to build ML-based solutions efficiently. This includes, for example, organizing the data (e.g. data discovery, data lineage, data quality), experimenting with models (e.g. notebooks, libraries, collaboration), one-click-deployment of a model (e.g. AI/ML platforms, infrastructure) and more. While many of these challenges are not that hard when working with small data, everything gets more complex & time-consuming when the data and scale is larger.

At the end of the conference we would like to invite all the attendees for the informal evening meeting at “Dekada” Club , which is located at the Grójecka 19/25, 02-021 Warszawa.