Snowflake Cloud Services Layer - Functions and Benefits

The services layer is the brain of Snowflake. It does a variety of Functions including

  • Authentication

  • Infrastructure management

  • Metadata management

  • Query parsing and optimization

  • Access control

Snowflake charms with its smart Metadata Collection

When the table is being loaded, Snowflake collects metadata about all the columns in the file at micro partition level and stores it in the metadata services layer. This is done automatically and done for all the columns

When you query against the table, certain queries are fullfilled by the cloud services layer itself, for example:

  • Select count(1) from Table; Since the total number of rows per micro partition and total number of partions per table information is available in the services layer. It will return the result directly without havint to scan the disk. How cool!
  • select max(join_dt) from Table; Since Snowflake knows the min and max dates for every partition, it will answer this query directly from the metadata
  • select min(join_dt) from Table; This is answered same way as above





MetricCamp.com