Le Sun, Aug 25, 2024 at 10:01:00AM +0200, Landry Breuil a écrit :
> Le Sun, Aug 25, 2024 at 09:36:26AM +0200, Landry Breuil a écrit :
> > Le Sun, Aug 25, 2024 at 03:30:56AM -0400, A Tammy a écrit :
> > > 
> > > On 8/25/24 3:07 AM, Landry Breuil wrote:
> > > > Le Thu, Aug 22, 2024 at 10:04:45AM +0200, Landry Breuil a écrit :
> > > >> Le Thu, Aug 22, 2024 at 09:40:23AM +0200, Landry Breuil a écrit :
> > > >>> Le Wed, Aug 21, 2024 at 06:11:32PM +0200, Kirill A. Korinsky a écrit :
> > > >>>> On Wed, 21 Aug 2024 17:16:11 +0200,
> > > >>>> Landry Breuil <landry@openbsd.org> wrote:
> > > >>>>> Le Wed, Aug 21, 2024 at 05:00:21PM +0200, Landry Breuil a écrit :
> > > >>>>>> Le Wed, Aug 21, 2024 at 03:49:45PM +0200, Kirill A. Korinsky a écrit :
> > > >>>>>>> ports@,
> > > >>>>>>>
> > > >>>>>>> Here a ping from another diff from my
> > > >>>>>> can you explain the dance about dropping/copying tests in pre-test ?
> > > >>>>>> why was that needed ? isn't doing MODPY_PYTEST_ARGS = --forked enough ?
> > > >>>>> now that i've tested, this fails with a rather strange error (strange as
> > > >>>>> in "i dont understand why moving tests/ around helps") on all test files:
> > > >>>>>
> > > >>>> and I confirm that this is the problem I am overlooking by moving tests.
> > > >> it feels we're getting close.. but not yet.
> > > > this version with a symlink for the rust lib works, and it's probably what im
> > > > going to commit. fixing the tests requiring a throwaway synapse test instance
> > > > is left for future work.
> > > >
> > > > Index: Makefile
> > > > ===================================================================
> > > > RCS file: /cvs/ports/net/synapse/Makefile,v
> > > > diff -u -r1.82 Makefile
> > > > --- Makefile    31 Jul 2024 16:01:52 -0000      1.82
> > > > +++ Makefile    25 Aug 2024 07:04:41 -0000
> > > > @@ -72,10 +72,20 @@
> > > >  TEST_DEPENDS = ${FULLPKGNAME}:${BUILD_PKGPATH} \
> > > >                 devel/py-mock${MODPY_FLAVOR} \
> > > >                 devel/py-parameterized${MODPY_FLAVOR} \
> > > > +               devel/py-test-forked${MODPY_FLAVOR} \
> > > >                 www/py-jwt${MODPY_FLAVOR}
> > > >  
> > > >  do-configure:
> > > >         @${MODCARGO_configure}
> > > > +
> > > > +MODPY_PYTEST_ARGS =    --forked
> > > > +
> > > > +# some tests fail, but they need a previously running synapse process, not just a database
> > > > +# E synapse.storage.prepare_database.UpgradeDatabaseException: Uninitialised database:
> > > > +# run the main synapse process to prepare the database schema before starting worker processes.
> > > > +# make sure that the rust library is found
> > > > +pre-test:
> > > > +       ln -sf ${WRKSRC}/build/lib.openbsd-${OSREV}-${ARCH}-cpython-${MODPY_MAJORMINOR}/synapse/synapse_rust.abi3.so ${WRKSRC}/synapse/
> > > >  
> > > >  # to generate rust modules.inc:
> > > >  #  make modcargo-gen-crates and modcargo-gen-crates-licenses
> > > 
> > > 
> > > I encountered almost exactly the same kind of bug for devel/py-uvloop.
> > > The solution was to move or delete the original sources folder, like so
> > > -
> > > http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/devel/py-uvloop/Makefile?rev=1.2&content-type=text/x-cvsweb-markup
> > > 
> > > pre-test:
> > > 	cd ${WRKSRC} && test -d uvloop && mv uvloop uvloop-bk || test -d uvloop-bk
> > > 
> > > This was because python tries to find the package to import in the local
> > > folder before it tries to find it in the PYTHONPATH environment vars
> > > locations.
> > > 
> > > Because the python tests run after the fake target has been completed,
> > > the move should be fine and shouldn't affect the PLIST generation.
> > 
> > yeah, i also figured that out after finding how nix runs the tests in
> > their synapse package, they rm -Rf synapse before starting tests.
> > 
> > interestingly, i also realized that upstream doesn't run tests via
> > pytest, but via twisted.trial, which might work around the "needs a
> > running instance of synapse first" issue.
> > 
> > currently trying with:
> > 
> > # this symlink makes sure that the rust library is found, since by default the source is used to run the tests
> > pre-test:
> >        ln -sf ${WRKSRC}/build/lib.openbsd-${OSREV}-${ARCH}-cpython-${MODPY_MAJORMINOR}/synapse/synapse_rust.abi3.so ${WRKSRC}/synapse/
> > 
> > do-test:
> >        cd ${MODPY_TEST_DIR} && ${SETENV} ${ALL_TEST_ENV} ${MODPY_BIN} -m twisted.trial tests
> > 
> > this produces a more verbose output with all tests names printed, but maybe
> > more tests succeed.
> 
> running tests with twisted.trial doesnt require pytest-forked, and only
> 3 tests fail:
> 
> tests.handlers.test_user_directory.UserDirectoryTestCase.test_search_punctuation
> twisted.trial.unittest.FailTest: Sequences differ: ['@user-a2:test'] != ['@user-a1:test']
> 
> tests.storage.test_room_search.EventSearchInsertionTest.test_null_byte
> twisted.trial.unittest.FailTest: 0 != 1
> 
> tests.storage.test_room_search.MessageSearchTest.test_sqlite_search
> twisted.trial.unittest.FailTest: 0 != 1 : expected 'quick brown' to match 'the quick brown fox jumps over the lazy dog'
> 
> FAILED (skips=303, failures=3, successes=3162)
> 
> the other skipped tests are for missing deps (or unsupported/not yet
> working tests):
> - OIDC
> - saml2
> - pyICU
> - jaeger_client
> - Postgres
> - authlib
> 
> so.. i don't know which is best. Keep using pytest --forked as it's more
> integrated within the portstree, or run the tests the same way upstream
> does ?

comparing the results with pytest:

=========================== short test summary info ============================
FAILED tests/app/test_openid_listener.py::FederationReaderOpenIDListenerTests::test_openid_listener_0
FAILED tests/app/test_openid_listener.py::FederationReaderOpenIDListenerTests::test_openid_listener_1
FAILED tests/app/test_openid_listener.py::FederationReaderOpenIDListenerTests::test_openid_listener_2
FAILED tests/app/test_openid_listener.py::FederationReaderOpenIDListenerTests::test_openid_listener_3
FAILED tests/handlers/test_user_directory.py::UserDirectoryTestCase::test_search_punctuation
FAILED tests/replication/test_federation_ack.py::FederationAckTestCase::test_federation_ack_sent
FAILED tests/storage/test_rollback_worker.py::WorkerSchemaTests::test_not_upgraded_current_schema_version_with_outstanding_deltas
FAILED tests/storage/test_rollback_worker.py::WorkerSchemaTests::test_not_upgraded_old_schema_version
FAILED tests/storage/test_rollback_worker.py::WorkerSchemaTests::test_rolling_back
FAILED tests/storage/test_room_search.py::EventSearchInsertionTest::test_null_byte
FAILED tests/storage/test_room_search.py::MessageSearchTest::test_sqlite_search
ERROR tests/federation/transport/server/test__base.py::test_disconnect
ERROR tests/http/test_servlet.py::test_disconnect
ERROR tests/replication/http/test__base.py::test_disconnect
ERROR tests/test_server.py::test_disconnect
= 11 failed, 3154 passed, 303 skipped, 205 warnings, 4 errors in 3730.21s (1:02:10) =

Landry