Add wp-tester for smoke testing across PHP and WordPress versions

[pfefferle]

> test:smoke
> wp-tester test


 ██     ██ ██████      ████████ ███████ ███████ ████████ ███████ ██████
 ██     ██ ██   ██        ██    ██      ██         ██    ██      ██   ██
 ██  █  ██ ██████         ██    █████   ███████    ██    █████   ██████
 ██ ███ ██ ██             ██    ██           ██    ██    ██      ██   ██
  ███ ███  ██             ██    ███████ ███████    ██    ███████ ██   ██

│
●   Test Run
│  
    PHP 7.4 + WP 6.5 - WordPress Tests
    ✓ should boot WordPress without errors 0ms
      WordPress
      ✓ create a post 371ms
      ✓ should load wp-admin 3.03s
    PHP 8.2 + WP latest - WordPress Tests
    ✓ should boot WordPress without errors 0ms
    PHP 8.3 + WP latest - WordPress Tests
    ✓ should boot WordPress without errors 0ms
    PHP 7.4 + WP 6.5 - Plugin Smoke Tests
    ✓ should boot WordPress without errors 1ms
      Plugin
      ✓ should be active 1.11s
    PHP 8.2 + WP latest - Plugin Smoke Tests
    ✓ should boot WordPress without errors 0ms
    PHP 8.3 + WP latest - Plugin Smoke Tests
    ✓ should boot WordPress without errors 0ms

  ✓ 15 passed
  ✗ 0 failed

  15 tests in 30.98s

  Legend: ✓ passed  ✗ failed  ○ skipped  ◔ pending  ◆ other

Proposed changes:

• Add wp-tester for running smoke tests in WordPress Playground environments
• Configure tests for PHP 7.4/8.2/8.3 and WordPress 6.5/latest matrix
• Use existing blueprint to activate the plugin during tests
• Add npm run test:smoke command for local testing

Other information:

• Have you written new tests for your changes, if applicable?

Testing instructions:

• Run npm run test:smoke locally
• Verify WordPress boot tests pass across all environments
• Verify plugin activation tests pass with blueprint

Changelog entry

• Automatically create a changelog entry from the details below.

Changelog Entry Details

Significance

• Patch
• Minor
• Major

Type

• Added - for new features
• Changed - for changes in existing functionality
• Deprecated - for soon-to-be removed features
• Removed - for now removed features
• Fixed - for any bug fixes
• Security - in case of vulnerabilities

Message

Add wp-tester for smoke testing plugin activation across PHP and WordPress versions.

[pfefferle]

@jeherve pointed me to wp-tester, so I ran some experiments. I quite like it, at least for local smoke tests!!

/cc @bgrgicak @akirk

[bgrgicak]

Thanks for trying WP Tester @pfefferle!

I'm taking a look at why WP Tester can't correctly detect PHPUnit in your plugin and will let you know once I have a fix, in case you want to try it.

[pfefferle]

thanks @bgrgicak !

I think I found the issue: bgrgicak/wp-tester#167

[bgrgicak]

I just released @wp-tester/cli@0.1.2 that includes your bootstrap path fix.
I tried testing wordpress-activitypub, and only a few tests fail due to authorization and publicKey issues.

I had to remove your Blueprints from the config as they try to install the plugin from WordPress.org, and that conflicts with WP Tester, which mounts the local plugin and activates it.

FYI, the test result output isn't great, and I plan to significantly improve it in the next release.

{
	"$schema": "https://raw.githubusercontent.com/bgrgicak/wp-tester/trunk/packages/config/src/schema.json",
	"projectType": "plugin",
	"environments": [
		{
			"name": "PHP 7.4 + WP 6.5",
			"php": [
				"7.4"
			],
			"wp": [
				"6.5"
			]
		},
		{
			"name": "PHP 8.2 + WP latest",
			"php": [
				"8.2"
			],
			"wp": [
				"latest"
			]
		},
		{
			"name": "PHP 8.3 + WP latest",
			"php": [
				"8.3"
			],
			"wp": [
				"latest"
			]
		}
	],
	"tests": {
		"plugin": "activitypub",
		"wp": true,
		"phpunit": {
			"phpunitPath": "vendor/bin/phpunit",
			"configPath": "phpunit.xml.dist",
			"testMode": "unit"
		}
	}
}

[pfefferle]

Updated based on @bgrgicak's feedback:

• Removed blueprints from environments (they conflict with WP Tester's local plugin mounting)
• Added PHPUnit configuration for running unit tests
• Added wp-tester.json to .distignore
• Rebased on latest trunk to resolve conflicts

[pfefferle]

@bgrgicak if I remove the blueprint, then the expected [] to include 'activitypub' always fails. That is why I added it initially.

[bgrgicak]

@bgrgicak if I remove the blueprint, then the expected [] to include 'activitypub' always fails. That is why I added it initially.

This is because your local plugin directory is named differently from the plugin slug (activitypub).
You can either set the virtual file system path to match the slug by adding "projectVFSPath": "/wordpress/wp-content/plugins/activitypub", to the config, or change the slug to wordpress-activitypub.

I still need to find a good way for WP Tester to deal with situations where the plugin directory on the host machine doesn't match the plugin slug.