Discussion notes during Jeremy Jones visit in Grenoble on 14-15th may with Laurent and Guillaume, then in Calern with Denis, Frédéric and Jean-Michel on 15-16th

Inline notes:

Summary of actions and associated priorities/milestones listed at the end of the notes

Observing Planning:

• Sharing array & instrument configuration is important : update baseline solutions, array, instrument specific (update MIRC-X setup) etc...
  • [ ] P0-C1 Setup access to the CHARA gitlab (https://gitlab.chara.gsu.edu/) for JMMC staff - request sent to Nils
  • JMMC maintains: http://apps.jmmc.fr/~swmgr/AsproOIConfigurations/
    • CHARA configuration should follow the beam order:
      • [ ] P1-C2 provide the new period (CHARA 2018A) with proper configurations (stations ordered by beams)
      • VEGA 2018A configuration lists the offered baselines + preferred PoPs (+ beams) that best fits science use cases
      • [ ] P1-C3 provide the correct (default) beams per instrument configuration (like VEGA does)
      • [ ] P1-J1 update Aspro2's configuration
    • New User preferences: interferometer / period / instrument to be used as default values
  • Denis: in some cases, there are discrepancies between ASPRO2 & CHARA plan observability (same config / PoPs): to be studied in details on concrete cases (related to the reference cart ?)
    • [ ] P1-V1 : provide detailed concrete cases of discrepancies in Pops btw Aspro and CHARA Plan for check
• Aspro2 is used in many different modes: preparation (with more or less constraints), scheduling programming, observing
  • Missing features compared to CharaPlan (see http://www.chara.gsu.edu/tutorials/timing-information )
    • cart position plot against the time :
      • helps to define POPS and observability range (1 cart is the reference, so 1 delay-line is fixed (at a specific position within 0 - 44m) for the CAL-SCI-CAL sequence (30mins at least ?)
      • How easy is it to solve cart positions ? How stable is the algorithm in terms of maintenance (inputs / parameters) ?
      • In CharaPlan the user can select the reference cart, but not in the Aspro2 GUI (VEGA considers always the first telescope of the configuration as the reference cart)
      • [ ] P3-J2 improve the GUI to define scope / beam / pop association in Aspro2 like in CharaPlan (combo boxes)
      • An option could be to expose how the best pop computation is performed in Aspro2 / and adapt the best pop algorithm (interactive) to take into account other constraints (shorter PoPs, reference cart...)
    • Alternative: use SAMP to better integrate ASPRO2 with CharaPlan ? to quickly check the cart positions more fluently (avoid filling GUI twice) and avoid to port specific code into ASPRO
    • [ ] P1-J3 Change default Twilight used at night limit to -12 deg for CHARA or all interferometers ?
    • Offer default / preferred interferometer (or order) and then change some default values depending on the instrument (UTC instead of LST @CHARA eg.)
• CHARA is operated in different modes (depending on the control knowledge of the observer)
• Data exchange between Aspro2 and outside world (A2... python tool)
  • Coming support of AO / FT / Guide stars in ASPRO2: CHARA has the same need (Lab AO stars, CHECK stars)
  • currently support of exporting most information shown in the aspro user setting (limited to one target) to XML (A2P2 approach)
  • [ ] P3-J4 develop an extension to export more content of the asprox file ? first all targets (displayed) like star lists
    • add PoPs info in best-!PoPs mode
    • add UTC next to LST Intervals in the xml file
    • convert to TXT file (CSV ?) to have an observing night plan. Jeremy sent a sample file for PAVO observations
    • later: export target notes + extra information (PI, program, groups)
• Target identification: HD number => CHARA number (to gather all info)
• No scheduler / queuing system in the control center: later (could use OB info)
• Missing information: PI name, Run ID / Program ID ?
  • use new specific fields or use the new group feature ?
  • possibly use the target notes or observation notebook (in ASPRO2) to store such extra information (instrument ETC info...)
• Observation logs:
  • how to retrieve / define the observed targets (with baseline / instrument modes) to track progress on large programs ?
    • new obs log table in ASPRO2 ? filled manually or completed by OIDB L0 queries
    • scan OIFits files ?
    • How to share such information ? using asprox files is cumbersome as it supposes to exchange files and possibly need to merge / keep in sync asprox files (changes + logs). Ideally a database is needed to store & retrieve up-to-date observation preparation & status
  • goal: mark / flag targets in ASPRO2 (observability / UV plan)
• How to better deal with multiple configurations ?
  • Denis & Jeremy are asking again if ASPRO2 could support multiple configurations : multiple baselines, CLIMB+MIRC or CLIMB+VEGA instruments, different PoPs setting per target... different instrument modes & settings (integration time) as it is difficult to work with multiple ASPRO2 instances (several asprox files not in sync, confusing):
  • ideally having several tabs (1 per setup) and also correctly managing the interop of the different tabs with the external applications (SearchCal for exemple).
  • or having the possibility to define & switch easily between setups would be great as it would allow to work on a single ASPRO2 instance (same target list)
  • It is a GUI issue: how to present such different observation setups ? Internally ASPRO2 already supports observation variants (multiple baselines)
  • To be studied: define several observation setup => SETUP1, SETUP2 ... and use target groups to associate targets to this setups (not exclusive i.e. a target may be observed using several setups ...)
  • (Best) PoPs issue: it depends on the all target list for now, but it should be set on target groups or on setups

Data Archive:

• CHARA get a file after 2015 with all logged metadata informations night per night
  • started to extract data from txt file into xml
    • The script is not working for all instruments, but soon will be.
    • Once complete, I will run the script on the archive from 2015-present, and then periodically update with current observing
    • Eventually, I will have the chara control software make these xml files and upload them automatically
    • Example script output:

<granule>
  <target_name>HD_41040</target_name>
  <s_ra>6.05760169444</s_ra>
  <s_dec>19.6905616667</s_dec>
  <t_exptime>0</t_exptime>
  <t_min>58155.131794</t_min>
  <t_max>58155.131794</t_max>
  <facility_name>CHARA</facility_name>
  <instrument_name>CLIMB_2</instrument_name>
  <nb_vis2>3</nb_vis2>
  <nb_t3>1</nb_t3>
  <em_min>2.2</em_min>
  <em_max>2.2</em_max>
  <obs_creator_name>Jeremy Jones</obs_creator_name>
  <datapi>Sandquist</datapi>
  <calib_level>0</calib_level>
</granule>

• • • • remarks:
        • [X] add - obs_id Done
        • why is t_exptime = 0 ? (and t_min should probably be different from t_max ) Only one time (t_max) is reported in the logs and in the data files (at least for Classic/CLIMB data)
        • could be added : nb_channels Done for Classic/CLIMB/JouFLU, more complicated for others
        • think about other ones like : instrument_mode, quality_level
    • http://oidb.jmmc.fr/show.html?id=116012
  • backlog can be ignored for fine details: keep only minimal metadata to contact the PI about observation on a specific target/date
  • /!\ think about providing key value that can be used in OIFits Header data for latter association
• A web page lists the program schedule with PI, Program Numbers etc... http://www.chara.gsu.edu/observers/observing-schedule
  • [ ] P1-C4 provide this data in a single file (xml e.g.)
  • Also, a webpage with Instrument PIs
• Observing report email can contains non standardised ProgNb (YYYY/PgrId)
• Data policy:
  • http://www.chara.gsu.edu/observers/data-policy-access
  • Is metadata confidential?
• OiDB improvements list:
  • [ ] P1-J5 give a pointer to the observing-schedule web page on every programId fields
  • [ ] P1-J6 support multiple programId (CHARA has such kind of record) ( or just concatenate ? )
  • [ ] P1-J7 add a free input metadata field to complete std ones with comments/quality etc... then request new std entries for most used anduseful ones
  • [ ] P1-J8 do not hide error report on upload failure by metadata upload
  • [ ] P2-J9 search by program ID

General discussed points:

• store a backup of OiDB on CHARA machines (few tens of MBytes TBC)
  • start with meta data + uploaded OIFITS
  • how to perform backup from CHARA side ? (no direct access for now)
    • [ ] P3-J10 provide a jmmc-backup script which downloads a single tar file (should be less than few GB)
      • and provide rotating time scale backup operations strategy : last 7 days, last 4 weeks, last 12 months + 1 tar per year ?
    • [ ] P3-C5 run jmmc-backup script each day with archival rotation
  • how JMMC retrieve previous backups ?
    • Just send an email to Jeremy ( waiting for a direct access in the future )
• backup CHARA's data on Grenoble site ?
  • raw data not under our scope at present time. Could be imagine with Grenoble University at cost.
  • talk later about reduced data : Spring/Fall 2019 – Reduced data accessible via the OIDB.
• A2P2 demo + ESO Observing portal ?
  • 2 options to start working in direction to CHARA interaction:
    • clone a2p2 to a2c...
    • handle another facility depending on the message sent by Aspro2
    • 2nd one is the prefered one - (-> new name proposal : Aspro 2 PreP)
    • [ ] P0-J11 provide a first implementation support for CHARA in A2P2
    • https://github.com/JMMC-OpenDev/a2p2
• every JMMC services and tools tour
  • bad calibrators feedbacks ?
  • New getstar service provides an up-to-date stellar diameter estimation (2.5M stars): http://www.jmmc.fr/getstar
• add current existing last data onto oidb-beta
  • draw the dataflow for future automatic update
• service monitoring
  • of JMMC webservice availability from CHARA machines
    • [ ] P3-J12 send jmmc-monitoring script samples and URLs lists to Jeremy.
  • of any CHARA service from JMMC side
    • [ ] P1-J13 monitoring CHARA's website availability
• News / general info :
  • CHARA wiki will move because WIKISPACES will shut down in next jully 31st
    • will move to dokuwiki
  • [ ] P0-J14 notify Gerard Van Belle about wikispace's shutdown for Olbin instance
• Current CHARA's archive size : 20 TB used today + 20~30 TB / year

VEGA specific points:

We discussed with Denis, Frédéric and Jean-Michel at Calern related to VEGA some new points:

• Long-term plan of getting OBs on CHARA
• How a2p2 works and possible future applications for CHARA
• Upcoming groups feature on aspro
• CHARA logs and my plans for getting them on OIDB
  • Short-term option:
    • [ ] P0-C6 CHARA produces std metadata for VEGA + addition details specific to VEGA in a separated file (instrument setup)
      • metadata are sent to oidb with a link to the complementary file(s)
  • detailed log files hosted on the CHARA web site or in the CHARA archive
• Missing information in L0 (obs logs): array setup (baseline / config at least), instrument setup (mode ?)
• [ ] P2-J15 OiDB could provide a way for datapi to edit their metadata, especially for tracking completion progress + indicate data quality & data processing status (DM can complete the list)

Email exchanges:

• encourage information exchanges on the jmmc-chara@jmmc.fr mailing list
• remind JMMC-OpenDev github repository : https://github.com/JMMC-OpenDev

Summary of actions and associated priorities/milestones:

task code : [' ' TODO - / started - X done] P<0 top priority - 3 no priority><C-hara/J-mmc/V-ega team>

P0 - End of july 2018

• [/] P0-C1 Setup access to the CHARA gitlab (https://gitlab.chara.gsu.edu/) for JMMC staff - request sent to Nils
• [ ] P0-C6 CHARA produces std metadata for VEGA + addition details specific to VEGA in a separated file (instrument setup)
• [X] P0-J11 provide a first implementation support for CHARA in A2P2
  • first version may be tested on a dedicated branch:

git clone https://github.com/JMMC-OpenDev/a2p2.git
cd a2p2 
pip install -U .
a2p2

• [X] P0-J14 notify Gerard Van Belle about wikispace's shutdown for Olbin instance

P1 - End of 2018

• [ ] P1-C2 provide the new period (CHARA 2018A) with proper configurations (stations ordered by beams)
• [ ] P1-C3 provide the correct (default) beams per instrument configuration (like VEGA does)
• [ ] P1-C4 provide this data in a single file (xml e.g.)
• [ ] P1-V1 : provide detailed concrete cases of discrepancies in Pops btw Aspro and CHARA Plan for check
• [ ] P1-J1 update Aspro2 configuration
• [ ] P1-J3 Change default Twilight used at night limit to -12 deg for CHARA or all interferometers ?
• [ ] P1-J5 give a pointer to the observing-schedule web page on every programId fields
• [ ] P1-J6 support multiple programId (CHARA has such kind of record) ( or just concatenate ? )
• [ ] P1-J7 add a free input metadata field to complete std ones with comments/quality etc... then request new std entries for most used anduseful ones
• [ ] P1-J8 do not hide error report on upload failure by metadata upload
• [ ] P1-J13 monitoring CHARA's website availability

P2 - Spring/Fall 2019

• [ ] P2-J15 OiDB could provide a way for datapi to edit their metadata, especially for tracking completion progress + indicate data quality & data processing status (DM can complete the list)
• [ ] P2-J9 search by program ID

P3 unscheduled / require specifications or discussions

• Not listed, see details in notes.