Forum der Vereinigung der Sternfreunde

Forum of the German Amateur Astronomy Association
Aktuelle Zeit: 18. Oktober 2018, 06:58:51 AM

Alle Zeiten sind UTC+02:00




Ein neues Thema erstellen  Auf das Thema antworten  [ 12 Beiträge ] 
Autor Nachricht
 Betreff des Beitrags: Data reduction - quality check
BeitragVerfasst: 03. August 2018, 12:25:46 PM 
Offline
Dauernutzer

Registriert: 06. Oktober 2016, 15:04:43 PM
Beiträge: 195
Hi,

I will answer on Robins comment and say some words about our reduction pipeline and quality check and discussions on our project.
Zitat:
Hi Christoph,

There seem to be some amateurs like Joan with experience in producing pro quality reduced spectra who might be interested in joining the campaign but have spectrographs which produce raw data that you cannot reduce yourself. Have you considered accepting their reduced data, perhaps with a comparison with the professional data for a particular night to check for quality? (I assume you will not be asking for raw data from the professional observatories :wink: )

Robin
Hey Robin, yes we are still rethinking some of our ideas on the data management.

Our question for raw data was based on our expierences with the LHires III. As you know this device isn't that stable due to flexure and temperature drift.
We are working on Linux and use ESO-MIDAS for most of our operations on the data. For reduction we wrote a script based pipeline to treat the data all in the same way.
One of the main reasons why we asked for raw data is to achieve good rv data.
Therefor we wrote also a script in which the telluric lines are used for recalibration. Therein first the tell. lines have to be measured. Then the descriptor start is recalculated with help of the mean value of deviation to rest wavelength from all measured lines and the spectrum shifted. After this one have to measure all used lines again for error estimation. At the end there is a camparison of the mean deviation values before and after and also the RMSE of before and after is calculated. The number of used lines (and the error from the fit on the calibration lamp data) is also shown. The RMSE after is written in the header as part error estimation. The second part comes from the RMSE of the fit on the calibration lamp data. This value is written in the Header at the first step of our pipeline.

If the telluric lines are good situated we are now able to reach a precision of ±2 km/s. As reference we use HiTran data. The values are between the real data from Hanuschik, 2006 and Coralie Neiner, out of the french amateur software. The maximum deviation in all reference data is less than 800 m/s.

Of course we need also the data for heliocentric correction (diurnal included).

If we now switch to 1D data, we should have to appoint the Header listings and looking for, if all software can provide the needed values. Also some more things are to fix to ensure a good data quality.

I hope you can see that our requests and the announcement letter is based on our knowledge and we communicated what we are able to do. We do not want use other amateur as byte providers and Anna and Indrek also not! There is not only a one way ticket in data exchange. We got also data from the professionals to work with them. And the announcement letter is written only by Malin and me! We got only hints from several professionals. Everyone who contribute anyhow to this project will be mentioned if he/she agrees.

By the way, Robin. Thank you for sharing your expierences of rv measurements with the LHires :wink:

Cheers
Christoph


Nach oben
   
 Betreff des Beitrags: Re: Data reduction - quality check
BeitragVerfasst: 04. August 2018, 13:41:02 PM 
Offline
Meister
Benutzeravatar

Registriert: 25. Oktober 2006, 23:43:13 PM
Beiträge: 717
Wohnort: Cumbria England
Hi Christoph,

I agree that the LHIRES needs special techniques to get high RV precision (though it looks like the standard reduced spectra from any instrument as submitted to BeSS for example could still be dropped into your pipeline before the recalibration using tellurics as is commonly done for LHIRES spectra)

The fibre fed echelle spectrographs do not have the same problem though and can give much higher RV precision than the LHIRES. I just think that by excluding amateurs using these instruments you are potentially losing the best quality amateur data.

If you can guarantee sufficient coverage with the contributors you have though then there is not a problem.

Cheers
Robin


Nach oben
   
 Betreff des Beitrags: Re: Data reduction - quality check
BeitragVerfasst: 04. August 2018, 17:47:31 PM 
Offline
Nutzer
Benutzeravatar

Registriert: 06. Dezember 2016, 11:37:47 AM
Beiträge: 41
Hi,

As far as I am concerned, I am in the same case as Joan: I have an echelle spectrograph (eShel from Shelyak Instrument) in a regulated temperature wine cellar (+/- 0.3 degrees Celsius) and the spectrum process for me is always the same with ISIS. (wavelength calibration with a Thorium / Argon lamp an RMS on each order around 0,002 Å). My configuration is very stable and all data are process the same manner.

I think that if you have doubts about the spectral calibration, you can check the reduced profil by looking at the atmospheric lines near the H Alpha line. (that's we do with submitted spectra in Bess database)

Advantages for the one who provides the spectrum : it has only one file to transmit

Benefit for spectrum verification: it is much faster and less cumbersome to manage than sending all the raw images.


Nach oben
   
 Betreff des Beitrags: Re: Data reduction - quality check
BeitragVerfasst: 05. August 2018, 15:49:23 PM 
Offline
Dauernutzer

Registriert: 06. Oktober 2016, 15:04:43 PM
Beiträge: 195
Hi Robin,
Zitat:
though it looks like the standard reduced spectra from any instrument as submitted to BeSS for example could still be dropped into your pipeline before the recalibration using tellurics as is commonly done for LHIRES spectra
Yes! That is right if all needed header listings are in the header. (In case of telluric recalibration there is only one listing necessary)
Zitat:
If you can guarantee sufficient coverage with the contributors you have though then there is not a problem.
This depends of course on the wether conditions! But it is better to have more data then less;)


Hi Olivier,
Zitat:
As far as I am concerned, I am in the same case as Joan: I have an echelle spectrograph (eShel from Shelyak Instrument) in a regulated temperature wine cellar (+/- 0.3 degrees Celsius) and the spectrum process for me is always the same with ISIS. (wavelength calibration with a Thorium / Argon lamp an RMS on each order around 0,002 Å). My configuration is very stable and all data are process the same manner.
I don't know ISIS. Is one able to define header listings and write data in these header listings (automaticly?). Is that an easy task in ISIS? So we may ask potentially contributors to write the headers we need.
The RMSE of your calibration fit for example we need for the recalibration and error estimation.

We had a look and comparison on the echelle data from Ulrich and Andrew with our long slit data. After recalibration of our long slit data there was a very good agreement in wavelength calibration.
But at the blue and red ends on the echelle we found deviations in normalisation the flux to one. In the middle area all fits.
Andrew told us that it is a little tricky because of the orders.

By the way, also a cool setup, Olivier :wink:

Cheers
Christoph


Nach oben
   
 Betreff des Beitrags: Re: Data reduction - quality check
BeitragVerfasst: 06. August 2018, 08:12:50 AM 
Offline
Dauernutzer

Registriert: 06. Oktober 2016, 15:04:43 PM
Beiträge: 195
Hi all,
these header listings would be the ("special") ones we need in 1-d data:

CAL_ERR: 1.88022E-03 /RMS error of fitting the calibration lines

OBSDATE: 2018-08-02T21:00:00 /mean date of observation

RESOLV: 13212 /spectral resolution= lambda/delta lambda at "wavelength"

SNR: 214 /calculated S/N of sqrt(electrons) at ~6450 Angstrom

VHEL: 18.2696 /heliocentric correction radial velocity (km/s)

MJD: 58332.900000000 /modified Julian Date of mean date of observation


And the telluric lines don't have to be removed!

Cheers
Christoph


Nach oben
   
 Betreff des Beitrags: Re: Data reduction - quality check
BeitragVerfasst: 06. August 2018, 23:00:36 PM 
Offline
Neueinsteiger

Registriert: 02. April 2017, 14:27:04 PM
Beiträge: 8
Hi all!

I started to write a blog on the VdS website for spectroscopy about our project. The idea is, that participants of the campaign and other interested people can have a look at what we do. It is also a kind of test-project, so that maybe other groups will present their projects on the website and enlive it.

I am sorry that at the moment just a german version exists - an english version will follow soon!

http://spektroskopie.vdsastro.de/testprojekt.html

Cheers
Malin


Nach oben
   
 Betreff des Beitrags: Re: Data reduction - quality check
BeitragVerfasst: 08. August 2018, 15:10:51 PM 
Offline
Meister
Benutzeravatar

Registriert: 25. Oktober 2006, 23:43:13 PM
Beiträge: 717
Wohnort: Cumbria England
Zitat:
these header listings would be the ("special") ones we need in 1-d data:
Yes unfortunately these are different keywords to those in the BeSS standard used in software like ISIS and BASS so the headers would need to be edited. (The information is there in some form in the ISIS header except for the photon count SNR and the lamp fit error. ISIS can be forced to output raw counts instead of scaling and it produces the lamp fit error in the output so these are available).

An example of the ISIS header below

SIMPLE T / File does conform to FITS standard
BITPIX -32 / Number of bits per data pixel
NAXIS 1 / Number of data axes
NAXIS1 7073 / length of data axis 1
CRVAL1 3805.5
CDELT1 0.5
VERSION 'ISIS V5.9.0' / Software version
OBJNAME 'AT2018blk' / Current name of the object
DATE-OBS '2018-05-14T22:54:12' / Date of observation start
EXPTIME 4555 / Total time of exposure
EXPTIME2 '15 x 300 s' / Exposure decomposition
BSS_INST 'C11_f5_ALPY600_ATK428' / Instrument
BSS_SITE 'THO robin@threehillsobservatory.co.uk' / Observation site
BSS_ITRP 532 / Typical resolving power
OBSERVER 'Leadbeater'
CUNIT1 'Angstrom' / Wavelength unit
CTYPE1 'Wavelength' / Axis type
Q_CAL '0 ' / Quality flag 4-excellent
CRPIX1 1 / Reference pixel
BSS_VHEL 0 / [km/s] Heliocentric speed
BSS_COSM 'Removed '
BSS_TELL 'None '
BSS_NORM 'None '
SPE_RPOW 532 / Spectral resolution power
JD-OBS 2458253.4543 / JD start observation
JD-MID 2458253.4807 / JD mid observation
JD-HEL 0 / JD heliocentric mid-obser
GEO_LONG -3.241 / Obs. geographic longitude
GEO_LAT 54.746 / Obs. geographic latitude
GEO_ELEV 135 / Obs. geographic elevation


Nach oben
   
 Betreff des Beitrags: Re: Data reduction - quality check
BeitragVerfasst: 16. August 2018, 12:07:05 PM 
Offline
Meister

Registriert: 01. August 2006, 16:20:19 PM
Beiträge: 835
Wohnort: Neumarkt i.d. Oberpfalz
Zitat:

Yes unfortunately these are different keywords to those in the BeSS standard used in software like ISIS and BASS so the headers would need to be edited. (The information is there in some form in the ISIS header except for the photon count SNR and the lamp fit error. ISIS can be forced to output raw counts instead of scaling and it produces the lamp fit error in the output so these are available).

Don't worry.
Thanks Astropy we can convert any keyword to a keyword Christoph wants to see. As long as the information is present.

günter

_________________
Im längsten Frieden spricht der Mensch nicht so viel Unsinn und Unwahrheit als im kürzesten Kriege. (Jean Paul)


Nach oben
   
 Betreff des Beitrags: Re: Data reduction - quality check
BeitragVerfasst: 17. August 2018, 21:10:21 PM 
Offline
Meister
Benutzeravatar

Registriert: 19. August 2006, 11:45:20 AM
Beiträge: 2713
Wohnort: 66907 Glan-Münchweiler
Hi,

Could I join the campagne?

Existiert there a paper to sign in?

Berthold

eShel ( from ~4200-~7000 Angström)


Nach oben
   
 Betreff des Beitrags: Re: Data reduction - quality check
BeitragVerfasst: 18. August 2018, 15:36:17 PM 
Offline
Dauernutzer

Registriert: 06. Oktober 2016, 15:04:43 PM
Beiträge: 195
Hi Berthold,

yes we wrote an announcement letter.

More information and the letter you will find here in the Forum:
viewtopic.php?f=32&t=5335

And here on the Blog (german):
http://spektroskopie.vdsastro.de/testprojekt.html

I will write you an E-Mail (hopefully the Forum-software works)

Cheers
Christoph


Nach oben
   
 Betreff des Beitrags: Re: Data reduction - quality check
BeitragVerfasst: 18. August 2018, 17:00:35 PM 
Offline
Meister
Benutzeravatar

Registriert: 19. August 2006, 11:45:20 AM
Beiträge: 2713
Wohnort: 66907 Glan-Münchweiler
Thank you......and where I should send my results?

Berthold


Nach oben
   
 Betreff des Beitrags: Re: Data reduction - quality check
BeitragVerfasst: 21. August 2018, 19:46:54 PM 
Offline
Neueinsteiger

Registriert: 02. April 2017, 14:27:04 PM
Beiträge: 8
Hi!

The english version of our blog on the website is online. http://spektroskopie.vdsastro.de/project-yhgs-en.html

Malin


Nach oben
   
Beiträge der letzten Zeit anzeigen:  Sortiere nach  
Ein neues Thema erstellen  Auf das Thema antworten  [ 12 Beiträge ] 

Alle Zeiten sind UTC+02:00


Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 1 Gast


Du darfst keine neuen Themen in diesem Forum erstellen.
Du darfst keine Antworten zu Themen in diesem Forum erstellen.
Du darfst deine Beiträge in diesem Forum nicht ändern.
Du darfst deine Beiträge in diesem Forum nicht löschen.
Du darfst keine Dateianhänge in diesem Forum erstellen.

Suche nach:
Gehe zu:  
cron
Powered by phpBB® Forum Software © phpBB Limited
Deutsche Übersetzung durch phpBB.de