2023/11/26

PHP Quality

要兼顧 PHP 開發品質,在 commit 之前可以用一些套件檢測,不管是否有符合 coding convention 或是有沒有語法上的錯誤,都可以預防性的監測。

composer require --dev phpstan/phpstan squizlabs/php_codesniffer friendsofphp/php-cs-fixer phpmd/phpmd phpunit/phpunit brianium/paratest

搭配 Makefile 可以減少每次執行要打的指令,此配置是針對 Laravel,可依照不同環境內容變動。

Makefile
fix-code-format:
	vendor/bin/php-cs-fixer fix --verbose

code-style-check:
	vendor/bin/phpstan analyse
	vendor/bin/phpcs --standard=PSR12 app tests
	vendor/bin/php-cs-fixer fix --dry-run
	vendor/bin/phpmd app,config,database,routes,tests text phpmd.xml

test: code-style-check
	vendor/bin/paratest --colors --processes 4 --runner=WrapperRunner
phpstan.neon
parameters:
  paths:
    - app
  excludePaths:
    - app/Http/Middleware/Authenticate.php
    - app/Providers/RouteServiceProvider.php

  level: 5

  parallel:
    processTimeout: 60.0
    maximumNumberOfProcesses: 32
    minimumNumberOfJobsPerProcess: 2

  checkMissingIterableValueType: false
.php-cs-fixer.php
<?php

use PhpCsFixer\Config;
use PhpCsFixer\Finder;

$rules = [
    'array_syntax' => ['syntax' => 'short'],
    'binary_operator_spaces' => ['default' => 'single_space'],
    'blank_line_after_namespace' => true,
    'blank_line_after_opening_tag' => true,
    'blank_line_before_statement' => true,
    'cast_spaces' => true,
    'class_definition' => true,
    'declare_equal_normalize' => true,
    'elseif' => true,
    'encoding' => true,
    'full_opening_tag' => true,
    'function_declaration' => true,
    'type_declaration_spaces' => true,
    'lowercase_cast' => true,
    'heredoc_to_nowdoc' => true,
    'include' => true,
    'indentation_type' => true,
    'lowercase_keywords' => true,
    'magic_constant_casing' => true,
    'method_argument_space' => true,
    'phpdoc_separation' => false,
    'native_function_casing' => true,
    'no_alias_functions' => true,
    'no_blank_lines_after_class_opening' => true,
    'no_blank_lines_after_phpdoc' => true,
    'multiline_whitespace_before_semicolons' => ['strategy' => 'no_multi_line'],
    'no_closing_tag' => true,
    'no_empty_phpdoc' => true,
    'no_empty_statement' => true,
    'no_leading_import_slash' => true,
    'no_leading_namespace_whitespace' => true,
    'no_multiline_whitespace_around_double_arrow' => true,
    'no_short_bool_cast' => true,
    'no_singleline_whitespace_before_semicolons' => true,
    'no_spaces_after_function_name' => true,
    'no_spaces_around_offset' => ['positions' => ['inside']],
    'spaces_inside_parentheses' => true,
    'no_trailing_comma_in_singleline' => true,
    'no_trailing_whitespace' => true,
    'no_trailing_whitespace_in_comment' => true,
    'no_unneeded_control_parentheses' => true,
    'no_unreachable_default_argument_value' => true,
    'no_unused_imports' => true,
    'no_useless_return' => true,
    'no_whitespace_before_comma_in_array' => true,
    'no_whitespace_in_blank_line' => true,
    'normalize_index_brace' => true,
    'not_operator_with_successor_space' => true,
    'object_operator_without_whitespace' => true,
    'ordered_class_elements' => true,
    'ordered_imports' => ['sort_algorithm' => 'alpha'],
    'phpdoc_indent' => true,
    'phpdoc_line_span' => ['const' => 'multi', 'method' => 'multi', 'property' => 'single'],
    'phpdoc_no_access' => true,
    'phpdoc_no_useless_inheritdoc' => true,
    'phpdoc_order' => true,
    'phpdoc_scalar' => true,
    'phpdoc_single_line_var_spacing' => true,
    'phpdoc_to_comment' => true,
    'phpdoc_trim' => true,
    'phpdoc_no_alias_tag' => ['replacements' => ['type' => 'var']],
    'phpdoc_types' => true,
    'phpdoc_var_without_name' => true,
    'no_mixed_echo_print' => ['use' => 'echo'],
    'self_accessor' => true,
    'short_scalar_cast' => true,
    'simplified_null_return' => true,
    'single_blank_line_at_eof' => true,
    'blank_lines_before_namespace' => true,
    'single_class_element_per_statement' => true,
    'single_import_per_statement' => true,
    'single_line_after_imports' => true,
    'single_quote' => true,
    'space_after_semicolon' => true,
    'standardize_not_equals' => true,
    'switch_case_semicolon_to_colon' => true,
    'switch_case_space' => true,
    'ternary_operator_spaces' => true,
    'trailing_comma_in_multiline' => ['elements' => ['arrays']],
    'trim_array_spaces' => true,
    'unary_operator_spaces' => true,
    'visibility_required' => true,
    'whitespace_after_comma_in_array' => true,
    'concat_space' => ['spacing' => 'one'],
    'single_space_around_construct' => true,
    'control_structure_braces' => true,
    'braces_position' => true,
    'control_structure_continuation_position' => true,
    'declare_parentheses' => true,
    'statement_indentation' => true,
];

$excludes = [
    'bootstrap/cache',
    'storage',
    'vendor',
    'node_modules',
];

$finder = PhpCsFixer\Finder::create()
                           ->in(__DIR__)
                           ->exclude($excludes)
                           ->notName('*.xml')
                           ->notName('*.yml');

$config = new PhpCsFixer\Config();
$config->setRiskyAllowed(true)
       ->setIndent('    ')
       ->setLineEnding("\n")
       ->setRules($rules)
       ->setUsingCache(true)
       ->setFinder($finder);

return $config;
phpmd.xml
<?xml version="1.0" encoding="UTF-8"?>
<ruleset name="Netask rule set"
    xmlns="http://pmd.sf.net/ruleset/1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://pmd.sf.net/ruleset/1.0.0 http://pmd.sf.net/ruleset_xml_schema.xsd"
    xsi:noNamespaceSchemaLocation="http://pmd.sf.net/ruleset_xml_schema.xsd">
<description>Netask code style rule set
</description>
<rule ref="rulesets/codesize.xml/CyclomaticComplexity"/>
<rule ref="rulesets/codesize.xml/NPathComplexity"/>
<rule ref="rulesets/codesize.xml/ExcessiveMethodLength"/>
<rule ref="rulesets/codesize.xml/ExcessiveClassLength"/>
<rule ref="rulesets/codesize.xml/ExcessiveParameterList"/>
<rule ref="rulesets/codesize.xml/ExcessivePublicCount">
    <properties>
        <property name="minimum" value="30"/>
    </properties>
</rule>
<rule ref="rulesets/codesize.xml/TooManyFields">
    <properties>
        <property name="maxfields" value="20"/>
    </properties>
</rule>
<rule ref="rulesets/codesize.xml/TooManyMethods"/>
<rule ref="rulesets/codesize.xml/ExcessiveClassComplexity">
    <properties>
        <property name="maximum" value="30"/>
    </properties>
</rule>
<rule ref="rulesets/controversial.xml"/>
<rule ref="rulesets/design.xml"/>
<rule ref="rulesets/naming.xml/ShortVariable"/>
<rule ref="rulesets/naming.xml/LongVariable">
    <properties>
        <property name="maximum" value="30"/>
    </properties>
</rule>
<rule ref="rulesets/naming.xml/ShortMethodName">
    <properties>
        <property name="minimum" value="2"/>
    </properties>
</rule>
<rule ref="rulesets/naming.xml/ConstructorWithNameAsEnclosingClass"/>
<rule ref="rulesets/naming.xml/ConstantNamingConventions"/>
<rule ref="rulesets/naming.xml/BooleanGetMethodName"/>
<rule ref="rulesets/unusedcode.xml/UnusedPrivateField" />
<rule ref="rulesets/unusedcode.xml/UnusedLocalVariable" />

<exclude-pattern>app/Console/Kernel.php</exclude-pattern>
<exclude-pattern>app/Services/Service.php</exclude-pattern>
<exclude-pattern>tests/TestCase.php</exclude-pattern>
</ruleset>
phpunit.xml
<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="./vendor/phpunit/phpunit/phpunit.xsd"
         bootstrap="vendor/autoload.php"
         colors="true"
>
    <testsuites>
        <testsuite name="Unit">
            <directory suffix="Test.php">./tests/Unit</directory>
        </testsuite>
        <testsuite name="Feature">
            <directory suffix="Test.php">./tests/Feature</directory>
        </testsuite>
    </testsuites>
    <coverage processUncoveredFiles="true">
        <include>
            <directory suffix=".php">./app</directory>
        </include>
    </coverage>
    <php>
        <server name="APP_ENV" value="testing"/>
        <server name="BCRYPT_ROUNDS" value="4"/>
        <server name="CACHE_DRIVER" value="array"/>
        <!-- <server name="DB_CONNECTION" value="sqlite"/> -->
        <!-- <server name="DB_DATABASE" value=":memory:"/> -->
        <server name="MAIL_MAILER" value="array"/>
        <server name="QUEUE_CONNECTION" value="sync"/>
        <server name="SESSION_DRIVER" value="array"/>
        <server name="TELESCOPE_ENABLED" value="false"/>
    </php>
</phpunit>

2023/09/23

Postman Interrupt Request By Environment-detecting

使用 postman 製作 API 測試時,有些案例可能不應該在 production 的情況執行,例如新增或修改一些會影響正式主機使用者的資料,但 postman 本身並沒有可以設定那些 API 在指定的環境時不可執行或者是警告執行,我們可以透過 Pre-request Script 撰寫 function 在 Collection 的層級,在各個 request 內放入限制。

Collection Pre-request Script

avoidExecuteFrom = (environment) => {
    const checkEnvironment = (value) => {
        if (pm.environment.name.includes(value)) {
            throw new Error(`forbidden in ${value}`)
        }
    };

    if (typeof environment === 'string') {
        checkEnvironment(environment);
    }

    if (Array.isArray(environment)) {
        environment.forEach(checkEnvironment);
    }
};

Request Pre-request Script

avoidExecuteFrom('staging');
avoidExecuteFrom(['production', 'staging'])

我的專案 environment 有三個配置,case-devcase-stagingcase-productionpm.environment.name 可以拿到 environment name,送出前偵測是否我們要避免執行的環境文字有包含在內,有的話便禁止送出。

2023/03/21

Mock Open Write On Python

在 python 的測試中,有時候 mock 只要 mock a.b.c 就可以成功,但我們使用了 with 的情況下,就必須借助 __enter__ 這個魔術方法來測試了,我們介紹兩種寫法。

my_open.py

def write(path, context):
    with open(path) as f:
        f.write(context)

test_my_open.py

import unittest
import my_open

from unittest.mock import mock_open, patch


class MyTestCase(unittest.TestCase):
    def test_my_open_1(self):
        with patch('builtins.open', mock_open()) as o:
            path = '/tmp/test.txt'
            context = 'hello'
            my_open.write(path, context)

            o.assert_called_once_with(path)
            o().write.assert_called_once_with(context)

    @patch('builtins.open')
    def test_my_open_2(self, mock_o: mock_open):
        path = '/tmp/test.txt'
        context = 'hello'
        my_open.write(path, context)

        mock_o.assert_called_once_with(path)
        handle = mock_o.return_value.__enter__.return_value
        handle.write.assert_called_once_with(context)


if __name__ == '__main__':
    unittest.main()

2023/02/07

Python unittest

最近在幫公司開發,使用 python 製作出 Windows 使用的小工具,順便 k 了一下單元測試,把遇到過的技巧做個筆記。

chan.py

import os
import shutil

import requests as requests


class Chan:
    def add(self, number1: int, number2: int) -> int:
        return number1 + number2

    def raise_method(self, result: str) -> str:
        if result != 'raise':
            return result

        raise Exception('exception raised')

    def a(self) -> str:
        return self.b()

    def b(self) -> str:
        return 'b'

    def move_file(self, from_path, to_path) -> object:
        return os.rename(from_path, to_path)

    def get_url(self, url) -> int:
        response = requests.get(url)

        return response.status_code

    def my_open(self, path) -> str:
        with open(path) as f:
            return f.read()

    def copy_twice(self, source: str, dest_dir: str) -> None:
        base_path = os.path.dirname(__file__)
        shutil.copyfile(source, os.path.join(base_path, dest_dir, 'first'))
        shutil.copyfile(source, os.path.join(base_path, dest_dir, 'second'))

    def scan_folder(self, path) -> list:
        files = []
        for item in os.listdir(os.path.join(path)):
            if os.path.isfile(os.path.join(path, item)):
                files.append(os.path.join(path, item))

        return files

tests/test_chan.py

import os.path
import unittest
from unittest.mock import patch, MagicMock, mock_open, call

from chan import Chan


def mock_response(*args, **kwargs) -> object:
    class Response:
        status_code = 500

    return Response()


class MyTestCase(unittest.TestCase):
    base_path: str

    def setUp(self) -> None:
        self.base_path = os.path.dirname(os.path.dirname(__file__))

    def test_add(self) -> None:
        chan = Chan()
        actual = chan.add(1, 2)
        self.assertEqual(3, actual)

    def test_raise_should_not_happened(self) -> None:
        chan = Chan()
        actual = chan.raise_method('test')
        self.assertEqual('test', actual)

    def test_raise_should_happened(self) -> None:
        chan = Chan()
        self.assertRaises(Exception, chan.raise_method, 'raise')

    def test_raise_error_message(self) -> None:
        chan = Chan()
        with self.assertRaises(Exception) as error:
            chan.raise_method('raise')

        self.assertEqual('exception raised', str(error.exception))

    @patch.object(Chan, 'b', MagicMock(return_value='c'))
    def test_mock_method(self) -> None:
        chan = Chan()
        actual = chan.a()
        self.assertEqual('c', actual)

    @patch.object(Chan, 'b')
    def test_mock_method_called_once_by_injection(self, mock_b: MagicMock) -> None:
        mock_b.return_value = 'c'
        chan = Chan()
        actual = chan.a()
        self.assertEqual('c', actual)
        self.assertTrue(mock_b.called)

    def test_mock_method_called_once_by_with(self) -> None:
        with patch.object(Chan, 'b') as check:
            check.return_value = 'c'
            chan = Chan()
            actual = chan.a()
            self.assertEqual('c', actual)
            check.assert_called_once()

    @patch('chan.os.rename', MagicMock(return_value='moved'))
    def test_os_method(self) -> None:
        chan = Chan()
        actual = chan.move_file('1.txt', '2.txt')
        self.assertEqual('moved', actual)

    @patch('chan.requests.get', MagicMock(side_effect=mock_response))
    def test_requests_get(self) -> None:
        chan = Chan()
        actual = chan.get_url('https://www.google.com')
        self.assertEqual(500, actual)

    @patch('chan.open', mock_open(read_data='ok'))
    def test_open(self) -> None:
        chan = Chan()
        actual = chan.my_open('path')
        self.assertEqual('ok', actual)

    @patch('chan.open', new_callable=mock_open, read_data='ok')
    def test_open_by_injection(self, m) -> None:
        chan = Chan()
        actual = chan.my_open('path')
        self.assertEqual('ok', actual)
        self.assertTrue(m.called)

    @patch('chan.shutil.copyfile')
    def test_copy_twice(self, mock_copyfile: MagicMock) -> None:
        chan = Chan()
        source = 'test.zip'
        chan.copy_twice(source, 'test_path')

        expected = [call(source, os.path.join(self.base_path, 'test_path', 'first')),
                    call(source, os.path.join(self.base_path, 'test_path', 'second'))]

        self.assertEqual(expected, mock_copyfile.call_args_list)
        self.assertEqual(2, mock_copyfile.call_count)

    @patch('chan.os.path.isfile')
    @patch('chan.os.listdir')
    def test_scan_folder(self, mock_listdir: MagicMock, mock_isfile: MagicMock) -> None:
        mock_listdir.return_value = ['dir1', 'file1', 'dir2', 'file2']
        mock_isfile.side_effect = [False, True, False, True]

        chan = Chan()
        test_dir = 'test_dir'
        actual = chan.scan_folder('test_dir')
        expected = [os.path.join(test_dir, 'file1'), os.path.join(test_dir, 'file2')]
        self.assertEqual(expected, actual)


if __name__ == '__main__':
    unittest.main()

2023/01/10

Powershell Note

Windows 的 cmd 不是很好用,我選擇使用 PoweShell 在 Windows 執行 command line,他還是沒有 Linux 的 cli 好用,但為了快速在 Window s 達到某些動作,還是來學一下基本指令並且做個筆記。

搜尋內文,類似 grep

ls -Path ./ -r | sls "WORD" | select Path -u

上面的效果是用 ls -r fetch 該目錄的所有檔案,再透過 sls 的 pipline 在每個撈出來的檔案中找尋內文符合的內容,最後利用 select Path -u 將重複的檔名過濾,在 PowerShell 裡面沒有大小寫之分。

搜尋檔案,類似 find

ls -Path ./ -r -Filter "WORD" -Name

在該目錄下搜尋檔案名稱。

2023/01/04

Vagrant SSH Issue

我的開發環境都是 base on Vagrant,某一天 Windows 更新後使用 vagrant ssh 時無法登入虛機內,本以為是更新到 2.3.4 後的 bug,因此發了這篇 issue,經過幾個回合的討論後,發現問題出在 Windows OpenSSH 更新後帶來的影響,因此要正常使用的話有幾個方法。

  1. 將 Vagrantfile 相關的資料目錄搬到你的個人目錄下,如 C:\Users\username\vagrant
  2. 使用 vagrant 自身的 embedded ssh,一般 command prompt 的話設定 SET VAGRANT_PREFER_SYSTEM_BIN=0,PowerShell 的話 key 入 $Env:VAGRANT_PREFER_SYSTEM_BIN=0,經測試都可以順利進入虛機。

2022/12/05

From Docker Virtual Network Connect To Solid MySQL Server

假設我們的 server 只有一台,PHP 環境為 docker container,而資料庫為實體機安裝,資料庫 3306 防火牆防堵不對外,但要允許從 Docker 環境內的服務存取,最簡單的方式就是防火牆讓 Docker 產生的 network interface 通過,下面我們來實做這件事。

今天示範環境為使用 Vagrant 起一台 Ubuntu jammy64,然後在裡面進行防火牆的測試。

sudo service ufw start
sudo ufw enable
sudo ufw allow ssh

記得開啟防火牆要先把 ssh 打開,不然就悲劇了,接著安裝 MySQL,docker 以及 docker compose 部分就麻煩自行安裝。

sudo apt install -y mariadb-server mariadb-client

安裝完畢後,我們來建立測試用資料庫以及帳號。

sudo mysql
create database laravel;
GRANT ALL PRIVILEGES ON `laravel`.* TO 'homestead'@'%' IDENTIFIED BY 'secret' WITH GRANT OPTION;

這邊權限開 % 沒關係,因為我們只會允許 docker network 指定的 network interface 進入而已。

docker-compose.yml
version: "3.9"

services:
  fpm:
    container_name: fpm
    build: .
    restart: always
    working_dir: /www
    extra_hosts:
      - "host.docker.internal:host-gateway"
    networks:
      - front_end
    volumes:
      - ./www/:/www

networks:
  front_end:
    driver_opts:
      com.docker.network.bridge.name: br-fpm
Dockerfile
FROM php:8.1-fpm

RUN apt-get update && apt-get install -y \
    git \
    curl \
    libpng-dev \
    libonig-dev \
    libxml2-dev \
    zip \
    unzip

ADD https://github.com/mlocati/docker-php-extension-installer/releases/latest/download/install-php-extensions /usr/local/bin/

RUN chmod +x /usr/local/bin/install-php-extensions && \
    install-php-extensions @composer pdo_mysql mbstring exif pcntl bcmath gd

WORKDIR /www

這是要安裝 Laravel 的最小環境,我們今天只是要測試連線所以其他的設定就先不理會了。

sudo docker compose up -d

等 image build 完以及 service 起來以後,我們來利用指令安裝一個 Laravel 的環境。

sudo docker compose exec fpm composer create-project laravel/laravel .

接著我們打開 .env 檔案設定資料庫連線內容。

DB_CONNECTION=mysql
DB_HOST=host.docker.internal
DB_PORT=3306
DB_DATABASE=laravel
DB_USERNAME=homestead
DB_PASSWORD=secret

這時來測試一下連線。

sudo docker compose exec fpm php artisan migrate

因為防火牆擋住了,所以 connection 會 hang 住,最終失敗,接著來開始防火牆設定。

sudo ufw allow in on br-fpm to any port 3306

接著再執行一次 migrate 的指令。

sudo docker compose exec fpm php artisan migrate

   INFO  Preparing database.

  Creating migration table ............................................................................................................... 10ms DONE

   INFO  Running migrations.

  2014_10_12_000000_create_users_table .................................................................................................... 9ms DONE
  2014_10_12_100000_create_password_resets_table .......................................................................................... 2ms DONE
  2019_08_19_000000_create_failed_jobs_table .............................................................................................. 3ms DONE
  2019_12_14_000001_create_personal_access_tokens_table ................................................................................... 4ms DONE

可以看到 migration 順利成功了,如果發生 connection refuse 的錯誤的話,記得把 MySQL 的 bind-address 設定為 0.0.0.0

iptable 的話用下方的指令應可達到一樣的效果。

iptables -A INPUT -i br-fpm -p tcp --dport 3306 -j ACCEPT
iptables -A INPUT -p tcp --dport 3306 -j DROP