python - Celery worker introspection to obtain hostname/nodename -
i want run py.test integration tests django 1.8 application full celery 3.1 setup, is, real queue (not celery_always_eager).
to that, need convince worker use server-based test db (postgresql) rather standard configured db (sqlite3, cannot talk second process).
to that, want modify databases setting in worker process before database first used. (i want keep sqlite other py.test tests normal non-test db interactive testing.)
to that, worker needs recognize running in test mode opposed 'normal' mode. indicate test mode, start worker different node names:
celery worker -a myapp --concurrency=1 -n myworker --loglevel=info
for normal mode versus
celery worker -a myapp --concurrency=1 -n mytestworker --loglevel=info
for test mode.
question: how can celery.py module in django application read out nodename (hostname, set -n
option) of worker?
there appear 3 processes involved (i have no further configuration concurrency beyond above =1
):
- the
celery.exe
(i on windows), - the worker itself, ,
- the single worker process (which relevant 1 me).
is correct?
there appear different signals celery sends each process:
celeryd_init
sentcelery.exe
,worker_init
sent worker,worker_process_init
sent worker process
is correct, too?
i added following celery.py
in django application:
import celery.signals def init_here(signal, sender): print("call %s(sender=%s)" % (signal, sender)) @celery.signals.celeryd_init.connect def celeryd_init(sender, instance, **kwargs): init_here("celeryd_init", sender) @celery.signals.worker_init.connect def worker_init(sender, **kwargs): init_here("worker_init", sender) @celery.signals.worker_process_init.connect def worker_process_init(sender, **kwargs): init_here("worker_process_init", sender)
when start worker in test mode indicated above, following output (among other things):
call celeryd_init(sender=celery@mytestworker) call worker_init(sender=celery@mytestworker) [2015-09-30 15:53:23,767: warning/mainprocess] celery@mytestworker ready. [2015-09-30 15:53:24,282: warning/worker-1] call worker_process_init(sender=none)
too bad! relevant process worker-1
not receive relevant information celery@mytestworker
. how can obtain hostname?
even if can work, not recommend using worker names detect whether in test mode or not. hack.
the way i've solved issue of knowing whether code running in test mode or not have different settings file called proj/test_settings.py
(whereas "regular" 1 @ proj/settings.py
), proj
project's name (same proj
below). have create setting determines whether in testing mode or not:
from .settings import * proj_testing = true
this variable set true test_settings
file. note possible have in test_settings.py
file lines override default values set in settings.py
. instance, have code there changes database configuration, , logging.
i arrange code launches tests set environment variable django_settings_module=proj.test_settings
. since celery workers used in testing launched code runs test, loading test settings , running proj_testing
true. check testing mode if need to, can this:
from django.conf import settings if settings.proj_testing: # need when testing. else: # else.
this method of doing things mirrors how set debug mode setting debug
setting true
.
Comments
Post a Comment