oom killed

Jan 24 06:09:00 ip-172-31-6-229 systemd[1]: Starting Clean php session files...
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540764] php7.1 invoked oom-killer: gfp_mask=0x24200ca, order=0, oom_score_adj=0
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540767] php7.1 cpuset=/ mems_allowed=0
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540773] CPU: 0 PID: 32751 Comm: php7.1 Not tainted 4.4.0-1060-aws #69-Ubuntu
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540774] Hardware name: Xen HVM domU, BIOS 4.2.amazon 08/24/2006
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540776]  0000000000000286 9bfaa3ce8896ec31 ffff88001a403848 ffffffff813fd3f3
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540779]  ffff88001a403a00 ffff880000810dc0 ffff88001a4038b8 ffffffff8120d4fa
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540781]  ffffffff81cc63b9 0000000000000000 ffffffff81e6b120 0000000000000206
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540783] Call Trace:
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540790]  [<ffffffff813fd3f3>] dump_stack+0x63/0x90
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540795]  [<ffffffff8120d4fa>] dump_header+0x5a/0x1c5
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540798]  [<ffffffff81194022>] oom_kill_process+0x202/0x3c0
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540800]  [<ffffffff81194449>] out_of_memory+0x219/0x460
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540804]  [<ffffffff8119a495>] __alloc_pages_slowpath.constprop.88+0x965/0xb00
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540806]  [<ffffffff8119a8b8>] __alloc_pages_nodemask+0x288/0x2a0
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540809]  [<ffffffff811e5d2d>] alloc_pages_vma+0xad/0x250
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540813]  [<ffffffff811a9ef0>] shmem_alloc_page+0x70/0xc0
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540817]  [<ffffffff810cd821>] ? __raw_callee_save___pv_queued_spin_unlock+0x11/0x20
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540820]  [<ffffffff814030d2>] ? radix_tree_lookup_slot+0x22/0x50
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540823]  [<ffffffff811ac9f5>] shmem_getpage_gfp+0x2c5/0x830
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540825]  [<ffffffff811ae4ce>] shmem_fault+0x6e/0x1e0
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540830]  [<ffffffff81337c34>] shm_fault+0x24/0x30
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540832]  [<ffffffff811bf7d6>] __do_fault+0x56/0xf0
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540834]  [<ffffffff811c2a3e>] handle_mm_fault+0x41e/0x1820
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540836]  [<ffffffff810cd821>] ? __raw_callee_save___pv_queued_spin_unlock+0x11/0x20
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540840]  [<ffffffff810ac2c6>] ? finish_task_switch+0x76/0x230
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540844]  [<ffffffff8181fc71>] ? __schedule+0x341/0x7f0
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540847]  [<ffffffff8106c891>] __do_page_fault+0x1a1/0x410
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540849]  [<ffffffff8106cb22>] do_page_fault+0x22/0x30
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540852]  [<ffffffff81826cd8>] page_fault+0x28/0x30
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540853] Mem-Info:
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540859] active_anon:98329 inactive_anon:8007 isolated_anon:0
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540859]  active_file:59 inactive_file:57 isolated_file:0
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540859]  unevictable:913 dirty:1 writeback:0 unstable:0
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540859]  slab_reclaimable:4332 slab_unreclaimable:5772
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540859]  mapped:14858 shmem:21006 pagetables:1711 bounce:0
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540859]  free:1126 free_pcp:30 free_cma:0
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540863] Node 0 DMA free:1900kB min:88kB low:108kB high:132kB active_anon:9376kB inactive_anon:1348kB active_file:24kB inactive_file:16kB unevictable:108kB isolated(anon):0kB isolated(file):0kB present:15988kB managed:15900kB mlocked:108kB dirty:0kB writeback:0kB mapped:2268kB shmem:2692kB slab_reclaimable:1224kB slab_unreclaimable:1016kB kernel_stack:128kB pagetables:228kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:268 all_unreclaimable? yes
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540868] lowmem_reserve[]: 0 455 455 455 455
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540871] Node 0 DMA32 free:2604kB min:2684kB low:3352kB high:4024kB active_anon:383940kB inactive_anon:30680kB active_file:212kB inactive_file:212kB unevictable:3544kB isolated(anon):0kB isolated(file):0kB present:507904kB managed:482536kB mlocked:3544kB dirty:4kB writeback:0kB mapped:57164kB shmem:81332kB slab_reclaimable:16104kB slab_unreclaimable:22072kB kernel_stack:2960kB pagetables:6616kB unstable:0kB bounce:0kB free_pcp:120kB local_pcp:120kB free_cma:0kB writeback_tmp:0kB pages_scanned:2548 all_unreclaimable? yes
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540876] lowmem_reserve[]: 0 0 0 0 0
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540878] Node 0 DMA: 47*4kB (UME) 65*8kB (ME) 39*16kB (UME) 18*32kB (UM) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1908kB
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540886] Node 0 DMA32: 115*4kB (UE) 118*8kB (UME) 69*16kB (UME) 3*32kB (UME) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2604kB
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540894] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540895] 21735 total pagecache pages
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540896] 0 pages in swap cache
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540898] Swap cache stats: add 0, delete 0, find 0/0
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540899] Free swap  = 0kB
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540900] Total swap = 0kB
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540901] 130973 pages RAM
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540902] 0 pages HighMem/MovableOnly
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540902] 6364 pages reserved
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540903] 0 pages cma reserved
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540904] 0 pages hwpoisoned
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540905] [ pid ]   uid  tgid total_vm      rss nr_ptes nr_pmds swapents oom_score_adj name
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540909] [  392]     0   392     9080      647      19       3        0             0 systemd-journal
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540912] [  430]     0   430    25743       58      16       3        0             0 lvmetad
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540914] [  890]     0   890     4031      215      12       3        0             0 dhclient
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540916] [ 1093]     0  1093     6512       52      19       3        0             0 atd
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540918] [ 1108]     0  1108    55572     2306      68       3        0             0 php-fpm5.6
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540920] [ 1115]     0  1115     7156       84      18       3        0             0 systemd-logind
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540922] [ 1120]     0  1120    68623      193      37       3        0             0 accounts-daemon
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540924] [ 1123]   104  1123    65159      439      29       4        0             0 rsyslogd
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540925] [ 1126]     0  1126     6933       71      19       3        0             0 cron
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540928] [ 1145]   107  1145    10754      155      25       3        0          -900 dbus-daemon
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540929] [ 1165]     0  1165    71558     5098      75       4        0             0 php-fpm7.1
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540931] [ 1168]     0  1168     1101       23       8       3        0             0 acpid
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540933] [ 1172]     0  1172   161035      296      33       4        0             0 lxcfs
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540935] [ 1180]     0  1180     6012       91      17       3        0             0 vsftpd
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540937] [ 1295]    33  1295    71652     1156      64       4        0             0 php-fpm7.1
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540939] [ 1296]    33  1296    71558      764      58       4        0             0 php-fpm7.1
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540941] [ 1307]     0  1307     3666       34      12       3        0             0 agetty
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540943] [ 1310]     0  1310     3620       37      12       3        0             0 agetty
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540945] [29238]     0 29238     3343       36      11       3        0             0 mdadm
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540947] [30145]   108 30145     6720       48      18       3        0             0 uuidd
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540949] [31020]   114 31020    27508      163      25       3        0             0 ntpd
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540951] [31165]     0 31165     1305       28       8       3        0             0 iscsid
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540953] [31166]     0 31166     1430      878       8       3        0           -17 iscsid
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540955] [31498]     0 31498    57357     2926      36       6        0          -900 snapd
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540957] [25564]     0 25564    16378      286      34       3        0         -1000 sshd
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540959] [ 6872]     0  6872     4973      487      14       3        0             0 mysqld_safe
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540961] [16895]     0 16895    41037      553      60       4        0             0 nginx
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540963] [16896]    33 16896    41282      904      62       4        0             0 nginx
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540965] [ 2466]   112  2466   149677    24069     107       3        0             0 mysqld
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540967] [ 2467]     0  2467     6203       95      16       4        0             0 logger
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540969] [ 3471]     0  3471    10530      381      23       3        0         -1000 systemd-udevd
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540971] [17747]    33 17747    66487    18759     111       3        0             0 php-fpm5.6
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540973] [23294]    33 23294    65432    17117     107       3        0             0 php-fpm5.6
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540975] [23439]    33 23439    65203    17175     106       3        0             0 php-fpm5.6
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540977] [ 7954]     0  7954    69273      173      38       3        0             0 polkitd
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540979] [31796]     0 31796     1126      174       8       3        0             0 apt.systemd.dai
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540981] [31803]     0 31803     1126      384       8       3        0             0 apt.systemd.dai
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540983] [31834]     0 31834    34399     8552      70       3        0             0 unattended-upgr
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540984] [31839]     0 31839    34399     8271      70       3        0             0 unattended-upgr
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540986] [32560]     0 32560    15967    10522      34       3        0             0 dpkg
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540988] [32593]     0 32593     1126      152       9       3        0             0 unattended-upgr
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540990] [32594]     0 32594     1126      397       8       3        0             0 invoke-rc.d
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540992] [32633]     0 32633     6223       67      15       3        0             0 systemctl
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540994] [32634]     0 32634    14649     2048      34       3        0             0 unattended-upgr
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540996] [32736]     0 32736     1126      177       8       4        0             0 sessionclean
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.540998] [32740]     0 32740     1126       24       8       4        0             0 sessionclean
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.541000] [32743]     0 32743     3632       43      10       4        0             0 sort
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.541001] [32744]     0 32744     3632       40      10       3        0             0 sort
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.541003] [32745]     0 32745     1126       24       7       4        0             0 sessionclean
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.541005] [32751]     0 32751    66445     2648      60       3        0             0 php7.1
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.541007] Out of memory: Kill process 2466 (mysqld) score 194 or sacrifice child
Jan 24 06:09:00 ip-172-31-6-229 kernel: [11712453.547784] Killed process 2466 (mysqld) total-vm:598708kB, anon-rss:96276kB, file-rss:0kB
Jan 24 06:09:00 ip-172-31-6-229 mysqld_safe: Number of processes running now: 0
Jan 24 06:09:00 ip-172-31-6-229 mysqld_safe: mysqld restarted
Jan 24 06:09:00 ip-172-31-6-229 mysqld: 190124  6:09:00 [Note] /usr/sbin/mysqld (mysqld 10.0.36-MariaDB-0ubuntu0.16.04.1) starting as process 311 ...
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [ERROR] mysqld: Out of memory (Needed 128909312 bytes)
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [Note] InnoDB: innodb_empty_free_list_algorithm has been changed to legacy because of small buffer pool size. In order to use backoff, increase buffer pool at least up to 20MB.
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [Note] InnoDB: Using mutexes to ref count buffer pool pages
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [Note] InnoDB: The InnoDB memory heap is disabled
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [Note] InnoDB: Compressed tables use zlib 1.2.8
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [Note] InnoDB: Using Linux native AIO
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [Note] InnoDB: Using CPU crc32 instructions
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [Note] InnoDB: Initializing buffer pool, size = 128.0M
Jan 24 06:09:01 ip-172-31-6-229 mysqld: InnoDB: mmap(139722752 bytes) failed; errno 12
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [ERROR] InnoDB: Cannot allocate memory for the buffer pool
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [ERROR] Plugin 'InnoDB' init function returned error.
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [Note] Plugin 'FEEDBACK' is disabled.
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [ERROR] Unknown/unsupported storage engine: InnoDB
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [ERROR] Aborting
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 
Jan 24 06:09:01 ip-172-31-6-229 systemd[1]: Started Clean php session files.
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 190124  6:09:01 [Note] /usr/sbin/mysqld: Shutdown complete
Jan 24 06:09:01 ip-172-31-6-229 mysqld: 
Jan 24 06:09:01 ip-172-31-6-229 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
Jan 24 06:09:01 ip-172-31-6-229 CRON[374]: (root) CMD (  [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi)
Jan 24 06:12:07 ip-172-31-6-229 systemd[1]: unattended-upgrades.service: Control process exited, code=exited status=1
Jan 24 06:12:07 ip-172-31-6-229 systemd[1]: Stopped Unattended Upgrades Shutdown.
Jan 24 06:12:07 ip-172-31-6-229 systemd[1]: unattended-upgrades.service: Unit entered failed state.
Jan 24 06:12:07 ip-172-31-6-229 systemd[1]: unattended-upgrades.service: Failed with result 'exit-code'.
Jan 24 06:12:07 ip-172-31-6-229 systemd[1]: Reloading.
Jan 24 06:12:07 ip-172-31-6-229 systemd[1]: Started ACPI event daemon.
Jan 24 06:12:08 ip-172-31-6-229 systemd[1]: Reloading.
Jan 24 06:12:08 ip-172-31-6-229 systemd[1]: Started ACPI event daemon.
Jan 24 06:12:09 ip-172-31-6-229 systemd[1]: Reloading.
Jan 24 06:12:09 ip-172-31-6-229 systemd[1]: Started ACPI event daemon.
Jan 24 06:12:09 ip-172-31-6-229 systemd[1]: Reloading.
Jan 24 06:12:09 ip-172-31-6-229 systemd[1]: Started ACPI event daemon.
Jan 24 06:12:09 ip-172-31-6-229 systemd[1]: Started Unattended Upgrades Shutdown.
Jan 24 06:12:13 ip-172-31-6-229 systemd[1]: Started Daily apt upgrade and clean activities.