Troubleshooting the AOC

Ohh Noo! Something’s gone wrong!


Figure: Dispraw image

Loops are not stable, in particular the Twt voltages wind up around the edges

The present WFS configuration has a rotation between the spot grid imaged inside the WFS onto the square array of CCD pixels.

If some factor causes the references to not match the open-loop measurement of this, an invisible mode will wind up in the system on the Twt integrator and cause the edges of the pupil to be ‘curled’. This makes the system go unstable.

If you run the closed loop and the system goes unstable from the edges in , particularly on a bright source, open the loops and dump out the tweeter commanded phase

IDL> res = rtc_retrieve_dumpdata(/grab, tweetercommp=p)
IDL> new_fancy_display_phase, p

If this looks (adjust the color scale) like the following, you’ve got issues.


Figure: Twt commands (inside pupil) when there is an un-referenced rotation between the spot grid and the CCD pixel grid.

This is most likely due to a stale dark file, but it could indicate an actual change in the internal rotation. This has happened at least twice (once at LLNL in 2010, once at UCSC in late 2011/early 2012).

Read on for more detail on why this occurs, how to diagnose if this is a temporary or long-term change, and how to fix it.

This means that the WFS is operating off null. If the gain on the WFS changes (due to spot size or much more importantly, background light) and we are operating with references (which we always are), the system will end up seeing spurious rotation.

Again: we know that the spots are rotated off. When we use references, we measure spots that are off-center (via the centroiding algorithm) and then subtract the known references to (hopefully) obtain zero rotation signal.

If the dark frame (see WFS_DARK_X) is not up to date, we can see either more background or fewer actual counts than actually exist after the dark subtraction processing. If we operate with more background light than is removed by the dark, this excess will lower the gain on the centroid calculation. This means we will measure less rotation than there actually is, and after reference subtraction, there will be a rotation error signal that is false.

Since this is uncorrectable optically, this will lead to a wind up of the quad-foil around the edges.


This is really a pain sometimes, as darks become stale on the order of minutes.

There is a very easy way to check to see if the dark is stale and there is rotation.

First, make sure we are using the regular references (see Managing References).

IDL> ult_check_ref_mode

If we aren’t, set the refs with

IDL> rtc_command_select_references, /meas

Now record what the light level is (note the SC power level). Take a WFS measurement to find the equivalent magnitude:

IDL> ult_check_light_level

Now run the code to measure the rotation.

IDL> print, ult_get_rotation()

This will print out the rotation (+ or - direction) in arcminutes. If this isn’t very small (< 0.1 or so), you need a fresh dark.

It’s very instructive to turn down the light source and take several more measurements. As the light source goes dimmer, the impact of the excess background light will become more pronounced. Don’t be surprised is by magnitude 8 or so you are measuring well over 0.5 arcminutes!


If you are taking measurements with a dim source, be sure that CENT_THRESH is sane. The ult_check_ref_mode code will keep you appraised of this. If you go to dim, you will end up meaureing all zeros, which is useless. Update the threshold with rtc_command_set_cent_thresh.

With this much rotation error, the system is not going to be stable.

The fix for this is easy - you take a fresh dark (see WFS Darks).

Now repeat the light-level rotation check again. You should have very small (< 0.1 arcmin) of rotation for the whole range of light levels.

If you do this and you have the same amount of rotation for all light levels, but it’s a significant level, it’s time to update the references. You can do this as described in the section above.

Lisa needs to finish this with final version of code...

Loops are not stable, less likely reasons

Verify the system gains. Use:

IDL> res = rtc_retrieve_dumpdata(gain=g)

to get the current gains in the control loop.

Check to see what the influence function filter is. If this has gotten mangled, the effective loop gains may be way off. This happened at UCSC in early 2012. Though you can read in TWT_INF_FUNC_MASK, there is no data dump to tell what is actually being used (e.g. ones or host). To verify, you have to analyze long time series of telemetry with IDL helpers.

To diagnose this, use the finaltests/ or finaltests/ codes to estimate the modal gains in closed loop. This works OK with just the light source, but it can be done nicely on a weak phase plate as well. If you find the modal gains are no longer uniform, you need to adjust the influence function filter appropriately.


Figure: Estimated closed-loop Fourier modal gains. The fact that the right panel is not flat indicates that the present influence function filter (TWT_INF_FUNC_MASK) was incorrect.

Wfr or Twt behaving strangely

Both Wfr and Twt electronics have suffered faults during I&T. Fuses on Wfr electronics have been blown twice, and several Wfr actuators were un operable for a significant period of time in Aug 2011 without being noticed. Poor connections caused some Twt actuators to be unresponsive in late 2011.

Sometimes these faults will be obvious. If a Twt actuator does not respond to voltage (i.e. it is ‘stuck’ at 0V), it will generate an obvious loss of intensity on a WFS image. Wfr actuators, due to their larger distance between them, will not make such an obvious (to the eye) change. You can just poke a specific actuator up and down by hand to see if it is responding. Running is the best option, though it does not test actuators right near the edge of the pupil.

Another thing to look for is if Twt actuators are forming patterns about the same spatial scale as a Wfr actuator. If a Wfr actuator is unresponsive, the Twt will be able to compensate for it to a significant degree, and that pattern will show up.

No light on WFS

  • Is everything turned on?
  • If using the supercontinuum source, check both the powerbar and the Source assembly - sometimes the laser overheats and shuts itself off. Also check the power level and attenuation.
  • If using the Tel Sim, is the OMSS shutter open? Is the ASU parked? If the ASU is partially extracted, it can block the beam.
  • If using the ASU, is it positioned correctly?
  • Is the SF closed all the way? If it is nominal (2.8 mm), a significant pointing change with the TT stage or the PNC or the InputFold can move the spot so far you get no light.
  • Are the Track assembly open loop models turned on? Are all mechanisms inited and datumed?