Test restructuring (#2237)

* Remove unused test util function * Refactoring ad and optiminterface tests to modules * Add SelectiveTests.jl * Rework tests CI GA * Switch test CI on Mac back to x64 * Remove coverage from CI. Improve matrix. * Wrap all tests in modules. * Remove unused test utils * Remove GA workflows for DynamicHMC and Numerical * Rename TuringCI GA to Tests * Fix test_args passing in CI * Fix for CI test matrix * Fixes to various test files * Add container.jl to test suite * Fix spacing around * in test includes * Split ad.jl and abstractmcmc.jl tests to separate CI jobs * Alphabetise imports in tests * In tests, use import X over using X: X * Add missing imports to abstractmcmc.jl tests * Add some missing imports to tests * Merge ad_utils.jl to ad.jl in tests * Merge testing_functions.jl into mh.jl in tests * Simplify test_utils Turn all of them into modules or merge them into other files that used to `include` them. * Add missing import to numerical_tests.jl * Update Project.toml (#2244) * Update README.md * Export adtype `AutoTapir` (#2236) * Export adtype `AutoTapir` * Update Project.toml * Fix missing AutoTapir (#2242) * Update Essential.jl * Update Project.toml * Drop support for ADTypes 0.2 (#2243) ADTypes 0.2 doesn't support AutoTapir yet. * Optimization improvements (#2221) * initial work on interface * Improving the Optimization.jl interface, work in progress * More work on Optimization.jl, still in progress * Add docstrings to Optimisation.jl * Fix OptimizationOptimJL version constraint * Clean up optimisation TODO notes * Relax OptimizationOptimJL version constraints * Simplify optimization imports * Remove commented out code * Small improvements all over in optimisation * Clean up of Optimisation tests * Add a test for OptimizationBBO * Add tests using OptimizationNLopt * Rename/move the optimisation test files The files for Optimisaton.jl and OptimInterface.jl were in the wrong folders: One in `test/optimisation` the other in `test/ext`, but the wrong way around. * Relax compat bounds on OptimizationBBO and OptimizationNLopt * Split a testset to test/optimisation/OptimisationCore.jl * Import AbstractADType from ADTypes, not SciMLBase * Fix Optimization.jl depwarning * Fix seeds in more tests * Merge OptimizationCore into Optimization * In optimisation, rename init_value to initial_params * Optimisation docstring improvements * Code style adjustments in optimisation * Qualify references in optimisation * Simplify creation of ModeResults * Qualified references in optimization tests * Enforce line length in optimization * Simplify optimisation exports * Enforce line legth in Optim.jl interface * Refactor away ModeEstimationProblem * Style and docstring improvements for optimisation * Add := test to optimisation tests. * Clarify comment * Simplify generate_initial_params * Fix doc references * Rename testsets * Refactor check_success * Make initial_params a kwarg * Remove unnecessary type constrain on kwarg * Fix broken reference in tests * Fix bug in generate_initial_params * Fix qualified references in optimisation tests * Add hasstats checks to optimisation tests * Extend OptimizationOptimJL compat to 0.3 Co-authored-by: Hong Ge <[email protected]> * Change some `import`s to `using` Co-authored-by: Tor Erlend Fjelde <[email protected]> * Change <keyword arguments> to kwargs... in docstrings * Add a two-argument method to OptimLogDensity as callable --------- Co-authored-by: Tor Erlend Fjelde <[email protected]> Co-authored-by: Hong Ge <[email protected]> * Update Project.toml * CompatHelper: bump compat for OptimizationOptimJL to 0.3 for package test, (keep existing compat) (#2246) Co-authored-by: CompatHelper Julia <[email protected]> --------- Co-authored-by: Markus Hauru <[email protected]> Co-authored-by: Tor Erlend Fjelde <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: CompatHelper Julia <[email protected]> * Set fail-fast: false for CI test matrix * Add a step to print matrix variables to tests Action * Fix typo in tests Action * ADTypes v0.2 compatibility for test restructuring (#2253) * Restore compat with ADTypes v0.2. Make AutoTapir export conditional. * Fix AutoTapir export in Essential.jl --------- Co-authored-by: Hong Ge <[email protected]> Co-authored-by: Tor Erlend Fjelde <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: CompatHelper Julia <[email protected]>
TuringLang · Jun 5, 2024 · 03d0e78 · 03d0e78
1 parent 87a040e
commit 03d0e78
Show file tree

Hide file tree

Showing 36 changed files with 621 additions and 507 deletions.
diff --git a/.github/workflows/DynamicHMC.yml b/.github/workflows/DynamicHMC.yml
diff --git a/.github/workflows/Numerical.yml b/.github/workflows/Numerical.yml
diff --git a/.github/workflows/Tests.yml b/.github/workflows/Tests.yml
@@ -0,0 +1,87 @@
+name: Tests
+
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ${{ matrix.os }}
+    continue-on-error: ${{ matrix.version == 'nightly' }}
+
+    strategy:
+      fail-fast: false
+      matrix:
+        test-args:
+          # Run some of the slower test files individually. The last one catches everything
+          # not included in the others.
+          - "essential/ad.jl"
+          - "mcmc/gibbs.jl"
+          - "mcmc/hmc.jl"
+          - "mcmc/abstractmcmc.jl"
+          - "mcmc/Inference.jl"
+          - "experimental/gibbs.jl"
+          - "mcmc/ess.jl"
+          - "--skip essential/ad.jl mcmc/gibbs.jl mcmc/hmc.jl mcmc/abstractmcmc.jl mcmc/Inference.jl experimental/gibbs.jl mcmc/ess.jl"
+        version:
+          - '1.7'
+          - '1'
+        os:
+          - ubuntu-latest
+          - windows-latest
+          - macOS-latest
+        arch:
+          - x64
+          - x86
+        num_threads:
+          - 1
+          - 2
+        exclude:
+          # With Windows and macOS, only run Julia 1.7, x64, 2 threads. We just want to see
+          # some combination work on OSes other than Ubuntu.
+          - os: windows-latest
+            version: '1'
+          - os: macOS-latest
+            version: '1'
+          - os: windows-latest
+            arch: x86
+          - os: macOS-latest
+            arch: x86
+          - os: windows-latest
+            num_threads: 1
+          - os: macOS-latest
+            num_threads: 1
+          # It's sufficient to test x86 with one version of Julia and one thread.
+          - version: '1'
+            arch: x86
+          - num_threads: 2
+            arch: x86
+
+    steps:
+      - name: Print matrix variables
+        run: |
+          echo "OS: ${{ matrix.os }}"
+          echo "Architecture: ${{ matrix.arch }}"
+          echo "Julia version: ${{ matrix.version }}"
+          echo "Number of threads: ${{ matrix.num_threads }}"
+          echo "Test arguments: ${{ matrix.test-args }}"
+      - uses: actions/checkout@v4
+      - uses: julia-actions/setup-julia@v2
+        with:
+          version: '${{ matrix.version }}'
+          arch: ${{ matrix.arch }}
+      - uses: actions/cache@v4
+        env:
+          cache-name: cache-artifacts
+        with:
+          path: ~/.julia/artifacts
+          key: ${{ runner.os }}-test-${{ env.cache-name }}-${{ hashFiles('**/Project.toml') }}
+          restore-keys: |
+            ${{ runner.os }}-test-${{ env.cache-name }}-
+            ${{ runner.os }}-test-
+            ${{ runner.os }}-
+      - uses: julia-actions/julia-buildpkg@latest
+      - name: Call Pkg.test
+        run: julia --color=yes --depwarn=yes --check-bounds=yes --threads=${{ matrix.num_threads }} --project=@. -e 'import Pkg; Pkg.test(; test_args=ARGS)' -- ${{ matrix.test-args }}
diff --git a/.github/workflows/TuringCI.yml b/.github/workflows/TuringCI.yml
diff --git a/Project.toml b/Project.toml
@@ -1,6 +1,6 @@
 name = "Turing"
 uuid = "fce5fe82-541a-59a6-adf8-730c64b5f9a0"
-version = "0.32.3"
+version = "0.33"
 
 [deps]
 ADTypes = "47edcb42-4c32-4615-8424-f2b9edc5f35b"

diff --git a/test/essential/ad.jl b/test/essential/ad.jl
@@ -1,5 +1,59 @@
+module AdTests
+
+using ..Models: gdemo_default
+using Distributions: logpdf
+using DynamicPPL: getlogp, getval
+using ForwardDiff
+using LinearAlgebra
+import LogDensityProblems
+import LogDensityProblemsAD
+using ReverseDiff
+using Test: @test, @testset
+using Turing
+using Turing: SampleFromPrior
+using Zygote
+
+function test_model_ad(model, f, syms::Vector{Symbol})
+    # Set up VI.
+    vi = Turing.VarInfo(model)
+
+    # Collect symbols.
+    vnms = Vector(undef, length(syms))
+    vnvals = Vector{Float64}()
+    for i in 1:length(syms)
+        s = syms[i]
+        vnms[i] = getfield(vi.metadata, s).vns[1]
+
+        vals = getval(vi, vnms[i])
+        for i in eachindex(vals)
+            push!(vnvals, vals[i])
+        end
+    end
+
+    # Compute primal.
+    x = vec(vnvals)
+    logp = f(x)
+
+    # Call ForwardDiff's AD directly.
+    grad_FWAD = sort(ForwardDiff.gradient(f, x))
+
+    # Compare with `logdensity_and_gradient`.
+    z = vi[SampleFromPrior()]
+    for chunksize in (0, 1, 10), standardtag in (true, false, 0, 3)
+        ℓ = LogDensityProblemsAD.ADgradient(
+            Turing.AutoForwardDiff(; chunksize=chunksize, tag=standardtag),
+            Turing.LogDensityFunction(vi, model, SampleFromPrior(), DynamicPPL.DefaultContext()),
+        )
+        l, ∇E = LogDensityProblems.logdensity_and_gradient(ℓ, z)
+
+        # Compare result
+        @test l ≈ logp
+        @test sort(∇E) ≈ grad_FWAD atol = 1e-9
+    end
+end
+
 @testset "ad.jl" begin
-    @turing_testset "adr" begin
+    @testset "adr" begin
         ad_test_f = gdemo_default
         vi = Turing.VarInfo(ad_test_f)
         ad_test_f(vi, SampleFromPrior())
@@ -50,7 +104,8 @@
         ∇E2 = LogDensityProblems.logdensity_and_gradient(zygoteℓ, x)[2]
         @test sort(∇E2) ≈ grad_FWAD atol = 1e-9
     end
-    @turing_testset "general AD tests" begin
+
+    @testset "general AD tests" begin
         # Tests gdemo gradient.
         function logp1(x::Vector)
             dist_s = InverseGamma(2, 3)
@@ -179,3 +234,5 @@
         @test ℓ_grad == ℓ_grad_compiled
     end
 end
+
+end
diff --git a/test/essential/container.jl b/test/essential/container.jl
@@ -1,3 +1,11 @@
+module ContainerTests
+
+import AdvancedPS
+using Distributions: Bernoulli, Beta, Gamma, Normal
+using DynamicPPL: @model, Sampler
+using Test: @test, @testset
+using Turing
+
 @testset "container.jl" begin
     @model function test()
         a ~ Normal(0, 1)
@@ -9,7 +17,7 @@
         x
     end
 
-    @turing_testset "constructor" begin
+    @testset "constructor" begin
       vi = DynamicPPL.VarInfo()
       sampler = Sampler(PG(10))
       model = test()
@@ -29,7 +37,7 @@
       @test DynamicPPL.get_num_produce(newtrace.model.f.varinfo) == 1
     end
 
-    @turing_testset "fork" begin
+    @testset "fork" begin
       @model function normal()
           a ~ Normal(0, 1)
           3 ~ Normal(a, 2)
@@ -48,3 +56,5 @@
       @test AdvancedPS.advance!(trace) ≈ AdvancedPS.advance!(newtrace) 
     end
 end
+
+end
diff --git a/test/experimental/gibbs.jl b/test/experimental/gibbs.jl
@@ -1,4 +1,12 @@
-using Test, Random, Turing, DynamicPPL
+module ExperimentalGibbsTests
+
+using ..Models: MoGtest_default, MoGtest_default_z_vector, gdemo
+using ..NumericalTests: check_MoGtest_default, check_MoGtest_default_z_vector, check_gdemo,
+    check_numerical
+using DynamicPPL
+using Random
+using Test
+using Turing
 
 function check_transition_varnames(
     transition::Turing.Inference.Transition,
@@ -187,3 +195,5 @@ end
         check_MoGtest_default_z_vector(chain, atol = 0.2)
     end
 end
+
+end