ExAliyunSls

Module VersionHex DocsTotal DownloadLicenseLast Updated

Description

ExAliyunSls is an Elixir SDK client for Aliyun SLS (阿里云日志服务). It allows you to push your logs to Aliyun Log Service, making them more convenient for statistics, visualization, and analysis.

Features

Installation

The package can be installed by adding :ex_aliyun_sls to your list of dependencies in mix.exs:

def deps do
  [
    {:ex_aliyun_sls, "~> 0.4"}
  ]
end

Logging Methods

In Elixir, you can log in the following ways:

Logger.debug "test1" #1
Logger.info fn -> "test2" end #2
Logger.info fn -> {"test3", [meta1: "meta1", meta2: "meta2", meta3: "meta3"]} end #3

Configuration

Aliyun SLS Configuration

Add your Aliyun SLS information to config/config.exs with the following options:

config :ex_aliyun_sls, :backend,
  endpoint: "YOUR SLS ENDPOINT",
  access_key_id: "YOUR ACCESS KEY ID",
  access_key: "YOUR ACCESS KEY",
  project: "YOUR SLS PROJECT NAME",
  logstore: "YOUR LOG STORE NAME",
  package_count: 100, # Default to 100
  package_timeout: 10_000, # Optional, Default to nil
  scheme: "https", # Default to "https"
  filtered_params: ["password", "token"] # Optional, for filtering sensitive data

Configuration Options:

Please ensure the access_key_id and access_key are correct and have the necessary permissions to write to your SLS project.

Logger Configuration

config :logger,
  backends: [
    {ExAliyunSls.LoggerBackend, :sls_log},
  ]

The atom :sls_log is just an identifier for the backend - you can use any atom you prefer. You can also add other logger backends alongside this one.

Configure which metadata to include in your logs:

config :logger, :sls_log,
  level: :info, # Optional, sets minimum log level
  metadata: [:pid, :module, :file, :line, :function, :application]

Metadata Options:

Log Level Filtering

You can set a minimum log level for the SLS backend:

config :logger, :sls_log,
  level: :info

This will only log messages with a level of :info or higher.

Phoenix Integration

Your logs through Phoenix endpoint are set by default by Plug.Logger . If you want to push them to Aliyun SLS, you should use our plug instead.

Replace the Plug Logger Handler

# This is the endpoint.ex in your phoenix project
# plug Plug.Logger
plug ExAliyunSls.Plug.Logger

With this configuration, your logs will be formatted as:

Logger.info fn ->
  {
    "GET: /login, status=200, duration=0.443ms",
    [
      duration: "0.443ms",
      status: 200,
      method: "GET",
      state: "set",
      request_path: "/login",
      params: "{your params will be formatted to json}"
    ]
  }
end

Your logs for plug will appear as "GET: /login, Sent 200 in 0.443ms", and it will push the metadata duration, method, request_path, status, state, and params to Aliyun SLS.

Filter Sensitive Parameters

If you have parameters that should not be logged, you can filter them by setting filtered_params in the config file:

config :ex_aliyun_sls, :backend,
  endpoint: "YOUR SLS ENDPOINT",
  access_key_id: "YOUR ACCESS KEY ID",
  access_key: "YOUR ACCESS KEY",
  project: "YOUR SLS PROJECT NAME",
  logstore: "YOUR LOG STORE NAME",
  package_count: 100,
  package_timeout: 10_000,
  filtered_params: ["password", "token", "credit_card", "secret"] # Add your filtered params here

Filtered parameters in HTTP requests will be replaced with "**" in the logs.

Embedded Page Integration

To check and search logs in the Aliyun SLS dashboard, you can add an embedded page to your own website.

Configuration

Create a role in the Aliyun RAM Access Control Console, set the proper attributes of the role and related permissions.

config :ex_aliyun_sls, :embed_page,
  access_key_id: "YOUR SLS ACCESS KEY ID",
  access_key_secret: "YOUR SLS ACCESS KEY SECRET",
  role_arn: "YOUR ROLE ARN",
  login_page: "YOUR LOGIN PAGE URL",
  destination: "YOUR DESTINATION URL"

Important: The above access_key_id and access_key_secret still come from the RAM user, which can be given specialized permissions to assume roles via the AliyunSTSAssumeRoleAccess authorization policy.

How to Use It

You can use ExAliyunSls.EmbedPage.get_url/7 to get the embedded page's URL:

get_url(access_key_id, access_key_secret, role_arn, login_page, destination_page, duration_seconds \\ 3600, role_session_name \\ "default")

Example Implementation

def get_embedded_dashboard_url do
  config = Application.get_env(:ex_aliyun_sls, :embed_page)
  
  {:ok, url} = ExAliyunSls.EmbedPage.get_url(
    config[:access_key_id],
    config[:access_key_secret],
    config[:role_arn],
    config[:login_page],
    config[:destination]
  )
  
  url
end

Advanced Usage

Batch Log Processing

The SDK automatically batches logs based on the package_count and package_timeout settings:

This approach optimizes network usage while ensuring logs are delivered in a timely manner.

Local Development

Use the following configuration to run the tests:

import Config

config :logger,
  backends: [
    {ExAliyunSls.LoggerBackend, :sls},
  ]
  
config :ex_aliyun_sls, :backend,
  endpoint: "...",
  access_key_id: "..."
  access_key: "..."
  project: "...",
  logstore: "...",
  package_count: 10,
  package_timeout: 5_000

config :ex_aliyun_sls, :embed_page,
  endpoint: "...",
  access_key_id: "...",
  access_key_secret: "...",
  role_arn: "..."

Copyright and License

Copyright (c) 2019 eDragonConnect

This work is free. You can redistribute it and/or modify it under the terms of the MIT License. See the LICENSE.md file for more details.