python - Celery worker introspection to obtain hostname/nodename -


i want run py.test integration tests django 1.8 application full celery 3.1 setup, is, real queue (not celery_always_eager).

to that, need convince worker use server-based test db (postgresql) rather standard configured db (sqlite3, cannot talk second process).

to that, want modify databases setting in worker process before database first used. (i want keep sqlite other py.test tests normal non-test db interactive testing.)

to that, worker needs recognize running in test mode opposed 'normal' mode. indicate test mode, start worker different node names:

celery worker -a myapp --concurrency=1 -n myworker --loglevel=info 

for normal mode versus

celery worker -a myapp --concurrency=1 -n mytestworker --loglevel=info 

for test mode.

question: how can celery.py module in django application read out nodename (hostname, set -n option) of worker?

there appear 3 processes involved (i have no further configuration concurrency beyond above =1):

  1. the celery.exe (i on windows),
  2. the worker itself, ,
  3. the single worker process (which relevant 1 me).

is correct?

there appear different signals celery sends each process:

  1. celeryd_init sent celery.exe,
  2. worker_init sent worker,
  3. worker_process_init sent worker process

is correct, too?

i added following celery.py in django application:

import celery.signals  def init_here(signal, sender):     print("call %s(sender=%s)" % (signal, sender))  @celery.signals.celeryd_init.connect def celeryd_init(sender, instance, **kwargs):     init_here("celeryd_init", sender)  @celery.signals.worker_init.connect def worker_init(sender, **kwargs):     init_here("worker_init", sender)  @celery.signals.worker_process_init.connect def worker_process_init(sender, **kwargs):     init_here("worker_process_init", sender) 

when start worker in test mode indicated above, following output (among other things):

call celeryd_init(sender=celery@mytestworker) call worker_init(sender=celery@mytestworker) [2015-09-30 15:53:23,767: warning/mainprocess] celery@mytestworker ready. [2015-09-30 15:53:24,282: warning/worker-1] call worker_process_init(sender=none) 

too bad! relevant process worker-1 not receive relevant information celery@mytestworker. how can obtain hostname?

even if can work, not recommend using worker names detect whether in test mode or not. hack.

the way i've solved issue of knowing whether code running in test mode or not have different settings file called proj/test_settings.py (whereas "regular" 1 @ proj/settings.py), proj project's name (same proj below). have create setting determines whether in testing mode or not:

from .settings import *  proj_testing = true 

this variable set true test_settings file. note possible have in test_settings.py file lines override default values set in settings.py. instance, have code there changes database configuration, , logging.

i arrange code launches tests set environment variable django_settings_module=proj.test_settings. since celery workers used in testing launched code runs test, loading test settings , running proj_testing true. check testing mode if need to, can this:

from django.conf import settings  if settings.proj_testing:   # need when testing. else:   # else. 

this method of doing things mirrors how set debug mode setting debug setting true.


Comments

Popular posts from this blog

html - Outlook 2010 Anchor (url/address/link) -

javascript - Why does running this loop 9 times take 100x longer than running it 8 times? -

Getting gateway time-out Rails app with Nginx + Puma running on Digital Ocean -