日々のあれこれφ(．．)

もっぱら壁打ち

alpコマンドでNginxのログ解析をする

「2021年はブログから離れない！」とか書いておきながら、1ヶ月ぶりの更新です🙂

理由は仕事が忙しかったからとか、プライベートが忙しかったからというのもあるのですが、その中でも時間はあったにも関わらず結局書けなかったのは、やっぱり書くことのハードルが知らず知らずの内に上がっていたからでした。なので初心にかえります。

alpコマンドでNginxのアクセスログを集計するやり方を知ったので、その覚書です。

alpコマンド

github.com

Access Log Profiler の意味らしいです。

NginxなどのWebサーバのアクセスログを

+-------+-----+------+-----+-------+-----+------------------+------------------------------------------------------------------------------+-------+-------+----------+-------+-------+-------+-------+--------+-----------+------------+--------------+------------+
| COUNT | 1XX | 2XX  | 3XX |  4XX  | 5XX |      METHOD      |                                     URI                                      |  MIN  |  MAX  |   SUM    |  AVG  |  P90  |  P95  |  P99  | STDDEV | MIN(BODY) | MAX(BODY)  |  SUM(BODY)   | AVG(BODY)  |
+-------+-----+------+-----+-------+-----+------------------+------------------------------------------------------------------------------+-------+-------+----------+-------+-------+-------+-------+--------+-----------+------------+--------------+------------+
|     1 |   0 |    1 |   0 |     0 |   0 | GET              | /isu/f46ad522-d196-487e-b666-c9b000c3d458/graph                              | 0.000 | 0.000 |    0.000 | 0.000 | 0.000 | 0.000 | 0.000 |  0.000 |   528.000 |    528.000 |      528.000 |    528.000 |
|     1 |   0 |    0 |   0 |     1 |   0 | GET              | /solr/                                                                       | 0.000 | 0.000 |    0.000 | 0.000 | 0.000 | 0.000 | 0.000 |  0.000 |   120.000 |    120.000 |      120.000 |    120.000 |
|     1 |   0 |    1 |   0 |     0 |   0 | GET              | /isu/c9add5fc-28ca-4eea-9b77-6cb90852508a/graph                              | 0.000 | 0.000 |    0.000 | 0.000 | 0.000 | 0.000 | 0.000 |  0.000 |   528.000 |    528.000 |      528.000 |    528.000 |
|     1 |   0 |    1 |   0 |     0 |   0 | GET              | /isu/023c8e56-4410-484d-b7fa-702d48188d3c                                    | 0.000 | 0.000 |    0.000 | 0.000 | 0.000 | 0.000 | 0.000 |  0.000 |   528.000 |    528.000 |      528.000 |    528.000 |
|     1 |   0 |    0 |   0 |     1 |   0 | -                | -                                                                            | 0.198 | 0.198 |    0.198 | 0.198 | 0.198 | 0.198 | 0.198 |  0.000 |   166.000 |    166.000 |      166.000 |    166.000 |
|     1 |   0 |    0 |   0 |     1 |   0 | GET              | /manager/text/list                                                           | 0.000 | 0.000 |    0.000 | 0.000 | 0.000 | 0.000 | 0.000 |  0.000 |   264.000 |    264.000 |      264.000 |    264.000 |
|     1 |   0 |    0 |   0 |     1 |   0 | POST             | /mifs/.;/services/LogService                                                 | 0.000 | 0.000 |    0.000 | 0.000 | 0.000 | 0.000 | 0.000 |  0.000 |   120.000 |    120.000 |      120.000 |    120.000 |
|     1 |   0 |    0 |   0 |     1 |   0 | GET              | /manager/html                                                                | 0.000 | 0.000 |    0.000 | 0.000 | 0.000 | 0.000 | 0.000 |  0.000 |   264.000 |    264.000 |      264.000 |    264.000 |

のような形に集計してくれます。

準備

準備として、nginxのログフォーマットをjson形式に対応させる必要があります。json以外にも、ltsv、pcapにも対応しています。

# nginx.conf

    log_format json escape=json '{"time":"$time_iso8601",'
                                '"host":"$remote_addr",'
                                '"forwardedfor":"$http_x_forwarded_for",'
                                '"req":"$request",'
                                '"status":"$status",'
                                '"method":"$request_method",'
                                '"uri":"$request_uri",'
                                '"body_bytes":$body_bytes_sent,'
                                '"referer":"$http_referer",'
                                '"ua":"$http_user_agent",'
                                '"request_time":"$request_time",'
                                '"cache":"$upstream_http_x_cache",'
                                '"runtime":"$upstream_http_x_runtime",'
                                '"rseponse_time":"$upstream_response_time",'
                                '"vhost":"$host"}';

    access_log  /var/log/nginx/access.log  json;

参考：alp/README.ja.md at main · tkuchiki/alp

これでnginxをrestartします。

$ tail -f /var/log/nginx/access.log
{"time":"02/Nov/2021:20:39:07 +0900","host":"147.192.34.114","forwardedfor":"","req":"GET /?jwt=eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MzU4NTQ5NDcsImlhdCI6MTYzNTg1MzE0NywiamlhX3VzZXJfaWQiOiJpc3Vjb24xIn0.CjOfNYVRt65cNgpMn3uxKxZiZ5PnRwX50NjHyf34SEIxsCtvmZ-aLL-GpZOJO8GAX1VvSAZeTZe7wdcNTT282w HTTP/2.0","status":"200","method":"GET","uri":"/?jwt=eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MzU4NTQ5NDcsImlhdCI6MTYzNTg1MzE0NywiamlhX3VzZXJfaWQiOiJpc3Vjb24xIn0.CjOfNYVRt65cNgpMn3uxKxZiZ5PnRwX50NjHyf34SEIxsCtvmZ-aLL-GpZOJO8GAX1VvSAZeTZe7wdcNTT282w","body_bytes":528,"referer":"http://localhost:5000/","ua":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Safari/537.36","request_time":"0.003","cache":"","runtime":"","rseponse_time":"0.004","vhost":"54.238.210.34"}
{"time":"02/Nov/2021:20:39:08 +0900","host":"147.192.34.114","forwardedfor":"","req":"GET /assets/index.23dac98b.js HTTP/2.0","status":"200","method":"GET","uri":"/assets/index.23dac98b.js","body_bytes":26667,"referer":"https://54.238.210.34/?jwt=eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MzU4NTQ5NDcsImlhdCI6MTYzNTg1MzE0NywiamlhX3VzZXJfaWQiOiJpc3Vjb24xIn0.CjOfNYVRt65cNgpMn3uxKxZiZ5PnRwX50NjHyf34SEIxsCtvmZ-aLL-GpZOJO8GAX1VvSAZeTZe7wdcNTT282w","ua":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Safari/537.36","request_time":"0.005","cache":"","runtime":"","rseponse_time":"0.004","vhost":"54.238.210.34"}
{"time":"02/Nov/2021:20:39:09 +0900","host":"147.192.34.114","forwardedfor":"","req":"GET /assets/index.144d8ca8.css HTTP/2.0","status":"200","method":"GET","uri":"/assets/index.144d8ca8.css","body_bytes":19066,"referer":"https://54.238.210.34/?jwt=eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MzU4NTQ5NDcsImlhdCI6MTYzNTg1MzE0NywiamlhX3VzZXJfaWQiOiJpc3Vjb24xIn0.CjOfNYVRt65cNgpMn3uxKxZiZ5PnRwX50NjHyf34SEIxsCtvmZ-aLL-GpZOJO8GAX1VvSAZeTZe7wdcNTT282w","ua":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Safari/537.36","request_time":"1.579","cache":"","runtime":"","rseponse_time":"0.008","vhost":"54.238.210.34"}

使い方

基本形

最小限のコマンドが以下です。

| alp json

$ cat /var/log/nginx/access.log | alp json
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+------------+------------+------------+------------+
| COUNT | 1XX | 2XX | 3XX | 4XX | 5XX | METHOD |                        URI                         |  MIN  |  MAX  |  SUM  |  AVG  |  P90  |  P95  |  P99  | STDDEV | MIN(BODY)  | MAX(BODY)  | SUM(BODY)  | AVG(BODY)  |
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+------------+------------+------------+------------+
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /                                                  | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 |  0.000 |    528.000 |    528.000 |    528.000 |    528.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /assets/index.23dac98b.js                          | 0.005 | 0.005 | 0.005 | 0.005 | 0.005 | 0.005 | 0.005 |  0.000 |  26667.000 |  26667.000 |  26667.000 |  26667.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /assets/index.144d8ca8.css                         | 1.579 | 1.579 | 1.579 | 1.579 | 1.579 | 1.579 | 1.579 |  0.000 |  19066.000 |  19066.000 |  19066.000 |  19066.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /assets/vendor.ee7444dd.js                         | 7.236 | 7.236 | 7.236 | 7.236 | 7.236 | 7.236 | 7.236 |  0.000 | 743417.000 | 743417.000 | 743417.000 | 743417.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /assets/logo_white.svg                             | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 |  0.000 |   3285.000 |   3285.000 |   3285.000 |   3285.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /api/isu/b82dcff4-8fae-4773-9956-22ab354adb7e/icon | 0.019 | 0.019 | 0.019 | 0.019 | 0.019 | 0.019 | 0.019 |  0.000 |   7994.000 |   7994.000 |   7994.000 |   7994.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | POST   | /api/auth                                          | 0.169 | 0.169 | 0.169 | 0.169 | 0.169 | 0.169 | 0.169 |  0.000 |      0.000 |      0.000 |      0.000 |      0.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /api/isu                                           | 0.068 | 0.068 | 0.068 | 0.068 | 0.068 | 0.068 | 0.068 |  0.000 |   3312.000 |   3312.000 |   3312.000 |   3312.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /api/isu/840e993f-3909-4970-8d9d-37b0a9ba0426/icon | 0.006 | 0.006 | 0.006 | 0.006 | 0.006 | 0.006 | 0.006 |  0.000 |   7994.000 |   7994.000 |   7994.000 |   7994.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /api/isu/4f2e409a-e5fb-48aa-92d6-56de38427e02/icon | 0.009 | 0.009 | 0.009 | 0.009 | 0.009 | 0.009 | 0.009 |  0.000 |  11209.000 |  11209.000 |  11209.000 |  11209.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /api/isu/ad39933b-791a-4f8f-90d7-0862212d71ad/icon | 0.012 | 0.012 | 0.012 | 0.012 | 0.012 | 0.012 | 0.012 |  0.000 |   6969.000 |   6969.000 |   6969.000 |   6969.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /api/isu/8fb74e6e-4f8d-4a33-9862-dbcdc0712c7e/icon | 0.013 | 0.013 | 0.013 | 0.013 | 0.013 | 0.013 | 0.013 |  0.000 |  14566.000 |  14566.000 |  14566.000 |  14566.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /api/isu/15063e10-26fc-4dac-ae3a-1f195e0c6c4a/icon | 0.014 | 0.014 | 0.014 | 0.014 | 0.014 | 0.014 | 0.014 |  0.000 |   9453.000 |   9453.000 |   9453.000 |   9453.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /api/isu/8469e6f6-0d29-4052-87cf-dd9bfde45014/icon | 0.015 | 0.015 | 0.015 | 0.015 | 0.015 | 0.015 | 0.015 |  0.000 |  20872.000 |  20872.000 |  20872.000 |  20872.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /api/isu/eb104b95-3532-4828-8550-89da717b9667/icon | 0.021 | 0.021 | 0.021 | 0.021 | 0.021 | 0.021 | 0.021 |  0.000 |  11209.000 |  11209.000 |  11209.000 |  11209.000 |
|     2 |   0 |   1 |   0 |   1 |   0 | GET    | /api/user/me                                       | 0.002 | 0.003 | 0.005 | 0.003 | 0.003 | 0.003 | 0.003 |  0.001 |     21.000 |     26.000 |     47.000 |     23.500 |
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+------------+------------+------------+------------+

同種のリクエストをまとめる

/api/isu/{uuid}/icon のような同種のリクエストをまとめることができます。以下のように正規表現でパターンを指定します。

--matching-groups= '{PATTERN}'

PATTERNはカンマ区切りで複数指定することもできます。

$ cat /var/log/nginx/access.log | alp json -m '/api/isu/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/icon'
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+------------+------------+------------+------------+
| COUNT | 1XX | 2XX | 3XX | 4XX | 5XX | METHOD |                                    URI                                     |  MIN  |  MAX  |  SUM  |  AVG  |  P90  |  P95  |  P99  | STDDEV | MIN(BODY)  | MAX(BODY)  | SUM(BODY)  | AVG(BODY)  |
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+------------+------------+------------+------------+
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /                                                                          | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 |  0.000 |    528.000 |    528.000 |    528.000 |    528.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /assets/index.23dac98b.js                                                  | 0.005 | 0.005 | 0.005 | 0.005 | 0.005 | 0.005 | 0.005 |  0.000 |  26667.000 |  26667.000 |  26667.000 |  26667.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /assets/index.144d8ca8.css                                                 | 1.579 | 1.579 | 1.579 | 1.579 | 1.579 | 1.579 | 1.579 |  0.000 |  19066.000 |  19066.000 |  19066.000 |  19066.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /assets/vendor.ee7444dd.js                                                 | 7.236 | 7.236 | 7.236 | 7.236 | 7.236 | 7.236 | 7.236 |  0.000 | 743417.000 | 743417.000 | 743417.000 | 743417.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /assets/logo_white.svg                                                     | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 |  0.000 |   3285.000 |   3285.000 |   3285.000 |   3285.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | POST   | /api/auth                                                                  | 0.169 | 0.169 | 0.169 | 0.169 | 0.169 | 0.169 | 0.169 |  0.000 |      0.000 |      0.000 |      0.000 |      0.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /api/isu                                                                   | 0.068 | 0.068 | 0.068 | 0.068 | 0.068 | 0.068 | 0.068 |  0.000 |   3312.000 |   3312.000 |   3312.000 |   3312.000 |
|     2 |   0 |   1 |   0 |   1 |   0 | GET    | /api/user/me                                                               | 0.002 | 0.003 | 0.005 | 0.003 | 0.003 | 0.003 | 0.003 |  0.001 |     21.000 |     26.000 |     47.000 |     23.500 |
|     8 |   0 |   8 |   0 |   0 |   0 | GET    | /api/isu/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/icon | 0.006 | 0.021 | 0.109 | 0.014 | 0.021 | 0.021 | 0.021 |  0.005 |   6969.000 |  20872.000 |  90266.000 |  11283.250 |
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+------------+------------+------------+------------+

ソート

カラムを指定して、ソートすることができます。

--sort {カラム名} --reverse

例えばリクエスト数が多いもの順に見たいときは、以下のようにできます。

$ cat /var/log/nginx/access.log | alp json -m '/api/isu/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/icon' --sort count -r
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+------------+------------+------------+------------+
| COUNT | 1XX | 2XX | 3XX | 4XX | 5XX | METHOD |                                    URI                                     |  MIN  |  MAX  |  SUM  |  AVG  |  P90  |  P95  |  P99  | STDDEV | MIN(BODY)  | MAX(BODY)  | SUM(BODY)  | AVG(BODY)  |
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+------------+------------+------------+------------+
|     8 |   0 |   8 |   0 |   0 |   0 | GET    | /api/isu/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/icon | 0.006 | 0.021 | 0.109 | 0.014 | 0.021 | 0.021 | 0.021 |  0.005 |   6969.000 |  20872.000 |  90266.000 |  11283.250 |
|     2 |   0 |   1 |   0 |   1 |   0 | GET    | /api/user/me                                                               | 0.002 | 0.003 | 0.005 | 0.003 | 0.003 | 0.003 | 0.003 |  0.001 |     21.000 |     26.000 |     47.000 |     23.500 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /                                                                          | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 |  0.000 |    528.000 |    528.000 |    528.000 |    528.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /assets/index.23dac98b.js                                                  | 0.005 | 0.005 | 0.005 | 0.005 | 0.005 | 0.005 | 0.005 |  0.000 |  26667.000 |  26667.000 |  26667.000 |  26667.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /assets/vendor.ee7444dd.js                                                 | 7.236 | 7.236 | 7.236 | 7.236 | 7.236 | 7.236 | 7.236 |  0.000 | 743417.000 | 743417.000 | 743417.000 | 743417.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /assets/logo_white.svg                                                     | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 |  0.000 |   3285.000 |   3285.000 |   3285.000 |   3285.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | POST   | /api/auth                                                                  | 0.169 | 0.169 | 0.169 | 0.169 | 0.169 | 0.169 | 0.169 |  0.000 |      0.000 |      0.000 |      0.000 |      0.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /api/isu                                                                   | 0.068 | 0.068 | 0.068 | 0.068 | 0.068 | 0.068 | 0.068 |  0.000 |   3312.000 |   3312.000 |   3312.000 |   3312.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /assets/index.144d8ca8.css                                                 | 1.579 | 1.579 | 1.579 | 1.579 | 1.579 | 1.579 | 1.579 |  0.000 |  19066.000 |  19066.000 |  19066.000 |  19066.000 |
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+------------+------------+------------+------------+

特定の時間以降（以前）のログを出力

--filters "Time >= '{日時}'"のような形で指定できます。

日時はログと同じフォーマットで指定します。

$ cat /var/log/nginx/access.log | alp json -m '/api/isu/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/icon' --filters "Time >= '2021-11-02T21:09:10+09:00'"
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+-----------+-----------+-----------+-----------+
| COUNT | 1XX | 2XX | 3XX | 4XX | 5XX | METHOD |                                    URI                                     |  MIN  |  MAX  |  SUM  |  AVG  |  P90  |  P95  |  P99  | STDDEV | MIN(BODY) | MAX(BODY) | SUM(BODY) | AVG(BODY) |
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+-----------+-----------+-----------+-----------+
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /api/isu                                                                   | 0.071 | 0.071 | 0.071 | 0.071 | 0.071 | 0.071 | 0.071 |  0.000 |  3312.000 |  3312.000 |  3312.000 |  3312.000 |
|     8 |   0 |   8 |   0 |   0 |   0 | GET    | /api/isu/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/icon | 0.006 | 0.019 | 0.099 | 0.012 | 0.019 | 0.019 | 0.019 |  0.005 |  6969.000 | 20872.000 | 90266.000 | 11283.250 |
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+-----------+-----------+-----------+-----------+

フォーマット

--format csv でcsv 形式に出力します。

他にも tsv や md （or markdown）でも出力できるみたいでした👀

結果をgitやslackで共有したいときに使えそう。

$ tail -n 10 /var/log/nginx/access.log | alp json -m '/api/isu/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/icon' --format csv
Count,1xx,2xx,3xx,4xx,5xx,Method,Uri,Min,Max,Sum,Avg,P90,P95,P99,Stddev,Min(Body),Max(Body),Sum(Body),Avg(Body)
1,0,1,0,0,0,GET,/assets/favicon.d0f5f504.svg,0.003,0.003,0.003,0.003,0.003,0.003,0.003,0.000,592.000,592.000,592.000,592.000
1,0,1,0,0,0,GET,/api/isu,0.071,0.071,0.071,0.071,0.071,0.071,0.071,0.000,3312.000,3312.000,3312.000,3312.000
8,0,8,0,0,0,GET,/api/isu/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/icon,0.006,0.019,0.099,0.012,0.019,0.019,0.019,0.005,6969.000,20872.000,90266.000,11283.250

dumpとload

--dump /tmp/alp.dump --load /tmp/alp.dump

dumpしたファイルは解析を行わずに解析結果を出力できるそうです。

$ tail -n 10 /var/log/nginx/access.log | alp json -m '/api/isu/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/icon' --dump /tmp/alp.dump
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+-----------+-----------+-----------+-----------+
| COUNT | 1XX | 2XX | 3XX | 4XX | 5XX | METHOD |                                    URI                                     |  MIN  |  MAX  |  SUM  |  AVG  |  P90  |  P95  |  P99  | STDDEV | MIN(BODY) | MAX(BODY) | SUM(BODY) | AVG(BODY) |
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+-----------+-----------+-----------+-----------+
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /assets/favicon.d0f5f504.svg                                               | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 |  0.000 |   592.000 |   592.000 |   592.000 |   592.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /api/isu                                                                   | 0.071 | 0.071 | 0.071 | 0.071 | 0.071 | 0.071 | 0.071 |  0.000 |  3312.000 |  3312.000 |  3312.000 |  3312.000 |
|     8 |   0 |   8 |   0 |   0 |   0 | GET    | /api/isu/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/icon | 0.006 | 0.019 | 0.099 | 0.012 | 0.019 | 0.019 | 0.019 |  0.005 |  6969.000 | 20872.000 | 90266.000 | 11283.250 |
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+-----------+-----------+-----------+-----------+

$ alp json --load /tmp/alp.dump
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+-----------+-----------+-----------+-----------+
| COUNT | 1XX | 2XX | 3XX | 4XX | 5XX | METHOD |                                    URI                                     |  MIN  |  MAX  |  SUM  |  AVG  |  P90  |  P95  |  P99  | STDDEV | MIN(BODY) | MAX(BODY) | SUM(BODY) | AVG(BODY) |
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+-----------+-----------+-----------+-----------+
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /assets/favicon.d0f5f504.svg                                               | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 | 0.003 |  0.000 |   592.000 |   592.000 |   592.000 |   592.000 |
|     1 |   0 |   1 |   0 |   0 |   0 | GET    | /api/isu                                                                   | 0.071 | 0.071 | 0.071 | 0.071 | 0.071 | 0.071 | 0.071 |  0.000 |  3312.000 |  3312.000 |  3312.000 |  3312.000 |
|     8 |   0 |   8 |   0 |   0 |   0 | GET    | /api/isu/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/icon | 0.006 | 0.019 | 0.099 | 0.012 | 0.019 | 0.019 | 0.019 |  0.005 |  6969.000 | 20872.000 | 90266.000 | 11283.250 |
+-------+-----+-----+-----+-----+-----+--------+----------------------------------------------------------------------------+-------+-------+-------+-------+-------+-------+-------+--------+-----------+-----------+-----------+-----------+

ちなみに /tmp/alp.dump の中身は以下のようになっていました。

- uri: /assets/favicon.d0f5f504.svg
  count: 1
  status1xx: 0
  status2xx: 1
  status3xx: 0
  status4xx: 0
  status5xx: 0
  method: GET
  response_time:
    max: 0.003
    min: 0.003
    sum: 0.003
    usepercentile: true
    percentiles:
    - 0.003
  request_body_bytes:
    max: 592
    min: 592
    sum: 592
    usepercentile: false
    percentiles: []
  response_body_bytes:
    max: 0
    min: 0
    sum: 0
    usepercentile: false
    percentiles: []
  time: ""

その他

  • --limit N： 出力する行数をN行に絞る

おわりに

できることは全部以下に載ってます。

alp/usage_samples.ja.md at main · tkuchiki/alp

多機能だけど、使い勝手はシンプルで公式ドキュメントが充実していて使いやすいなと感じました。

業務ではもっぱらモニタリングツールに転送したりしているので、こうやってアクセスログを直接見る機会はだいぶ減ったのですが、機会があれば活用していけたらと。

以下は製作者のzenn記事です。

zenn.dev