March 2, 2020

Continued Teaching

tyler775

Now I preparing to teach a student about Data Warehouses. These are technologies that aggregate data from multiple sources so they can be compared and analyzed for various purposes. They can:

  1. Hold data for a long period of time
  2. optimize operations for reading data
  3. hold data for long periods of time
  4. Hold data that may lag and not be updated in real-time

There are many types of data warehouses like Vertica, Teradata, Oracle, and IBM. There is Apache Hive, a new open-source warehouse and the main one my student and I will be going over for this session. It part of the larger Hadoop ecosystem.

*Hadoop: distributed computing framework for processing millions of records. The process for Hadoop goes like this:

  1. Store millions of records in multiple machines
  2. Run processes on multiple machines to crunch data
  3. Handle fault tolerance/machine crashes
  4. Hive stores data in Hadoop process (data stored in files - text, binary) and partitioned across machines to prevent data loss

Comments

Times shown in UTC

No comments yet.

You must be signed in to post a comment.