Azure Log Analytics - Some scribbles with Azure Files log queries (preview)

So, Azure Storage Accounts are probably somewhat known for everyone. If not, you can read more about them from below MS Docs link:

Storage account overview - Azure Storage | Microsoft Docs

This article is specifically tailored to Azure Files (since that's what I'm testing out):

Introduction to Azure Files | Microsoft Docs

So, back to the actual business.

I've been tinkering around with some Storage log analytics to get some simple data from Azure Files Shares. Since I'm a noob, this is the most I could come up with.

While other standard File Services offer robust monitoring for file operations, Azure Files probably leaves a lot to be desired for customers that have established setups already configured. If you are like me though, who has nothing configured beforehand, putting together at least something that can retrieve data is a welcome surprise.

So, how can I start out with this?

Firstly, you have to enable the logs via the storage account and send them to Log Analytics:

Monitoring Azure Files | Send to Log Analytics | Microsoft Docs

After this has been enabled for the desired Storage Account, you have to wait for a few days for the data to start flowing into the Log Analytics Workspace.

After this, you can move to actually running queries towards the data.

Queries that are done towards the Log Analytics logs are done via Kusto Query language.

More about Kusto can be found from the following MS link:

Kusto Query overview- Azure Data Explorer | Microsoft Docs

But for example, a quick scribble that will fetch all successful write operations within 90 minutes of running the query in a pie chart is as follows:

StorageFileLogs

| where Protocol == "SMB"

  and TimeGenerated >= ago(90m)

  and OperationName has "Write"

  and Uri !has "~$"

  and Uri !has ".tmp"

  and Uri !has "$"

| distinct AccountName, Uri

| summarize count() by AccountName

| render piechart 

So while that can be useful for getting general info from Share write distribution, finding the actually useful data might be more difficult.

One example I can quickly thing of is finding the top ten most used files within the last 7 days:

StorageFileLogs

| where Protocol == "SMB"

  and TimeGenerated >= ago(7d)

  and OperationName !has "Ioctl"

  and OperationName has "Read"

  and Uri !has "~$"

  and Uri !has ".tmp"

  and Uri !has "$"

| distinct AccountName, Uri, TimeGenerated, CallerIpAddress, OperationName

| summarize count() by Uri

| filter Uri !has ".pptx"

| top 10 by count_

While the data here contains all transaction records, it should show the most used files of the organization, or the files that most used by applications that also leverage the files.

These also cut off some obvious files that should not be counted by the uri, such as .tmp files or files with "$" in the path. These can be tailored to the individual needs of the data structure these queries are being used towards. Also filters out ".pptx" for example purposes from the results, but for example some obvious file names can be added to the filter if they are programmatic files that should not be there.

While all of these are still in preview, the logs do provide a lot of useful other info as well.

One of the most particular ones I've liked is the query to see if the storage operations are slow on the user or on the Azure end:

StorageFileLogs

| where Protocol == "SMB"

  and TimeGenerated >= ago(1d)

  and Uri !has "~$"

  and Uri !has ".tmp"

  and Uri !has "$"

| distinct OperationName, CallerIpAddress, AccountName, Uri, TimeGenerated, DurationMs, ServerLatencyMs

| extend ClientLatency = DurationMs - ServerLatencyMs 

| top 10 by ClientLatency

This lists the top 10 operations by supposed Client Latency. Client Latency is calculated roughly by the total duration and server latency data by simple substraction. As usual, these can be taken with a grain of salt, but do offer some nice info on what Gateway IP addresses show signs of slowness in the operations.

And another good example is a specific file path drilling query to files that have been modified the most on a folder structure (within 30 minutes):

StorageFileLogs

| where Protocol == "SMB"

  and Uri has "\\\\storageaccount.file.core.windows.net\\sharename\\subfolder\\"

  and TimeGenerated >= ago(30m)

  and OperationName has "Write"

| summarize count() by Uri

| top 10 by count_

The only special thing about the above query is that it requires the double backslashes to work. If you use the Uri as it's seen on the Log Analytics or as it's viewed normally, the query won't work.

Modifying these base query examples is rather easy, since the query language itself is rather simple to understand, and the Log Analytics query editor offers nice suggestions to configure the queries.

Do mind that these are the queries of an amateur query maker, so refining them should be easy for the ones that fiddle around with the Kusto queries daily.

However, hopefully these offer some insight and help with building some baseline queries for Azure Files management and alerts via Azure Alerting:

Monitoring Azure Files | Alerting | Microsoft Docs

Till next time...