Author Archives: loek

Variable variables in Painless and Ingest Node

Just a short note: it is possible to create variable (or dynamic) variables in both Painless scripts and the Ingest Node Set Processor. In Painless, you can do achieve this with the following notation: ctx[ctx.key_variable] = value Say you have two arrays that you’d like to ‘merge’ in Painless. So, the values of one of… Read More »

Here’s how to find out the optimal memory settings for your Hadoop cluster

Mature Relational Database Systems (RDBMSs) are very much self-optimizing. In fact, the capabilities of such a system to determine for itself how to execute the query in the most efficient way is so good that the pinnacle programming language of RDBMSs – the Structured Query Language (SQL) – almost completely focuses on what should be done rather than how it should be done. This is not as true in the world of emerging technology Hadoop. Hadoop demands a little more attention from the Engineers, Scientists and/or Administrators to make it fly. Luckily, there is a tool to help out and it will show you how to find the optimal values for your cluster in minutes!