Stitching: microscope scan path vs. DP

Microscope goes col-by-col, then row-by-row.
So we have less horizontal drifts than vertical ones.
So we can use greedy on horizontal and DP on vertical.

Stitching: image alignment major failure

We used "Greedy" method that only align newly introduced images with the ones already in place, regardless images that haven't been put in the array.

In our case, 690 rows of images, one-pixel vertical error per row will cause an accumulated vertical error as large as one FOV!!

DP is necessary.

FOV chain

Chain start has to be found.

Start with lowest FOV (assume disc is placed in normal position):
[r_grid_max, c_grid_mid]

For each CC in FOV, try to load gridcc_next recursively;
If c_grid_next == c_grid, we found a valid start, which leads to its inner neighbor successfuly.
If c_grid_next not found, then we reached a deadend,
move to [r_grid_max-1, c_grid_mid]

sting removal: w/z not robust at all

give up on sting removal

w/z not robust.

and in/out pairing up should move to stitching-main

Interview: Tell me about yourself

Tell me about yourself (2 minutes)

1. About myself, I have professional and personal tracks.
2. I have been doing image and audio related research and development for the past 7 years.
3. Actual
   Experence: My most recent experience has been doing an innovative
   multi-disciplinary research that involves optics, image recognition and
   digital audio signal processing. 
4. What I gained: I've developed good R&D methodology, organized research and programming styles.
1. Research content: This work is unique; it requires to deal with a programmable white-light interferometer microscope, which involves mechanics and optics, and with millions of images that contain grooves at micrometer-to-nanometer level;
   all of them need to be recorgnized in the image domain yet eventually
   interpreted in the audio sense. This is the kind of work that needs
   both patience and imagination, which are both my strength. I gained
   interesting signal processing experience in connecting both domains.
1. My
   other strength is in R&D methodology: From various experience of
   working with new problems that involves massive data with little
   reference to resort to, I have cultivated an organized and cautious
   research behavior and programming style that works pretty well with
   exploration of ideas and constant evolution in algorithm implementation.
1. In a procedural research, I strive to do a depth-first style to try moving forward
   towards the end result in the early stage of research, and skip little
   bumps as fast as possible until a skeleton workflow is built up.
2. Then
   as the whole problem gets better understood during the course, I would
   then work on each step of the workflow or adapt the workflow according
   to any new surprises. 
3. For recorgnition problems, I would start
   with creating a meaningful and precise data
   visualization that helps me verify any change that my real work
   makes, and I use incremental tests on my extremely large datasets for
   any new idea or step in a workflow; 
4. To
   ensure any failure during long experiment running time, I have detailed
   temporary data storage and retrieval schemes, so even a power failure
   wouldn't cost me too much.
5. My coding style makes such
   evolutionary processes easier by using systematic naming conventions,
   template-functional design pattern, and I would tidy things up whenever
   new functions come into the whole system with necessary code
   refactoring. As a result, my main client routinesseldom change.
6. I respect performance optimization in coding but I use them only after I have better understanding of the problem.
5. I
   have pretty good sense of presenting ideas, work, and very considerate
   about applying various design principles to my presentation more
   comprehensible and efficient.
6. I'm strong in independent problem-solving.
7. In
   various research projects and extra-curriculum work, I've also worked
   in teams, as a member and the coordinator as well. People often find me
   an easy-going and cooperative person.
8. When having to meet dealines, I'm very flexible and adaptible in my R&D approach;
   if brute-force works better in such a situation I would just use it.
   When the automation of the microscope wasn't working well, I manually
   scanned 700 images to meet the deadline.
9. What I'm looking for now is a challeging project that's more widely applicable, using my experience, knowledge and skills, and I'd like to join a strong team and have a positive impact on the R&D process.

Interview Memo: high level construction

Prepared topics

1. Myself: experience, training, knowledge, soft skills, not in CV
   
2. Past project: Handel, Webcast, Music Tech projects (screenshot, code samples)
   
3. Research: content, method, value
   
4. Company
   

Bottom line

1. Maybe I can't answer all question. But I can answer those that are not supposed to be failed.
2. I'm supposed to know my own research inside out.
   
3. Confidence
4. Decisive
5. don't lead the interview, let them guide you.

bottom 1D: sidewall based thinning

Problem

1. broken bottom
2. get the real 1D ones
3. simple features: median radius, lowest points, not satisfactory

Analysis

1. bottom = "lowest sidewall tip band"
2. from outer band to inner band, the extreme radial positions are sidewalls for sure?
3. If there is just one point, then no sidewall to analyze.
4. The acquired band is usually 2D, tangentially thick layered. So extracted 1D should be a vector instead of one single point.
5. Then the real 1D point, should be picked from the vector according to p-to-line dist.
6. #4 does the radial picking, #5 does tangential picking
7. Need to check the z-contour of bottom: under various resolution, they show certain trail of stylus tip across band, though extremely noisy.
   
8. cannot assume all bottom points are distributed across the groove center.
9. so we have to make sure which of them belong to which side of the walls.
10. ASSUMPTION: the 1D point is among the detected points.
    
    

Implementation

1. find center by in/out edge.
2. divide 2d band into in/out wall groups.
3. linear fitting on both sides,
   
4. find the cross as the center.
5. if one sidewall is missing, assume its delta_r/z with prev is proportional with the edges'.
   

New tertiary build: accumulated error in finding outer-edge pair

Problem

1. Line-equation algo introduces error.
2. This error accumulates along the iteration, if we start search from each (prev+1) location
3. Error is due to the "min-dist" scheme, which ignores the actual numerical tangential error.
   

Solution

1. There has to be limitation in tangential error.
2. What would be the threshold?

Major analysis

1. Outer/inner edges are noise signal.
2. We expect that they should be 1v1 relationship, while in reality they are not due to stings and other noises.
3. Tangential Redundancy: should be defined as "if two inner points find a common outer pair, then one of them is redundant."
4. In Case #3, preserve only the 1st inner point along the tangential direction.
5. Since inner signal has bigger tangential density, we need to remove inner-edge redundancy first.
6. Bottom has the most redundancy because it is the most 2D-ish among the tertiary edges.
7. Too strict line-equation would miss out bottom band completely.
8. Too loose line-equation would carry neighbor and select wrong band.
9. Bottom has to be found through 2D measure.
   
10. We need to try the sidewall-tip idea in 1D, and ensure no redundancy in the other 1D.
11. Bottom tangential redundancy can be removed by assuming bottom depends on both edges.
    
12. W2Z Reference should use "majority" instead of "mean".

Implementation

1. Do regular tertiary search based on inner edge.
2. Do redundancy check against outer-edge along the way instead of a pre-step.
3. After inner/outer pair determined, find bottom band with inner/outer line-equation.
4. For bottom, when using minimum point-to-line distance measure, should discard redundant result (pick the first non-overlap point from sorted distance list).
   

tertiary: bottom from in/out pair

Problem

1. line-equation: dist from bottom to in/disc-center line diverse. hard to threshold.
2. pair azimuth range lock: too narrow to catch anything

Solution

1. in-center line equation: with flexible CLOSENESS threshold
2. include previous point into the search process (only way to work)
   

update_neighbor: some new ccs are found having bottom

Cause

• a few noises recognized as bottom,
  
• they exist in ho area, i.e., not contained by any CC
  

Solution

• in btm_indexed_by_container(), remove noise btm, and redo cc_analysis for btm.
  

Pictures to take with microscope

1. grid
2. bottom with jags
3. 11X acquired image

bottom 1D from 2D: linear slope problematic

the prev linear slope algo needs to have both sidewalls to work.
if one side wall portion is missing then there is no way to find the true stylus tip

audio alignment: edge , bottom

linear interpolation to get same azimuths at the overlap between prev and cur FOV.CC.win.edge.
get dr, dz,
apply to cur FOV
merge cur with main thread

image alignment redesign

Used simpler and straightforward design to update FOV locations.
Dealt with too-large views when trying to display them; divide and conquer, currently only 2-piece display

image stitching revisit

Problem

1. inter-FOV z diff is always present.
2. align them in image domain or audio ?

Analysis

1. If z un-aligned in image, later update_neighbor() filtering will transfer wrong z values to neighbor-FOVs that don't align with the focus-FOV.
2. Stitching in image registration gives rough result that's visually satisfied.
3. Stitching in audio registration yields aurally satisfied refined stitching.
   

remove sidewall imp

1. for each bottom_thread(cc), 
2. - for each overlap_pair,
3. - - get median_a of each btm
4. - - get edges(a) from line_equation()
   
5. - - get valley-center(a) from edges(a)
6. - - get delta_r(center, median_a)
7. - - sidewall.append(bigger delta)
8. remove sidewalls and re-cc-analysis
9. for each bottom_thread(cc)
10. - for each non_overlap_pair,
11. - - #3-5
12. - - get delta_r(center, median_a) / width(a)
    
13. - - sidewall.append(delta > THRESH)
14. remove sidewalls and re-cc-analysis
15. update edge(3)
    

remove sidewall

Problem

1. Sidewall may not overlap with anything or slightly overlaps if there is no bottom in center area.

Solution

1. using center valley as reference, check out 1) for overlapped pairs, which one is closer to the center; 2) for non overlapped bottoms, threshold them with off-center ratio.
   

Presentation: lab meeting

Top level

1. Post scan: Workflow
2. Mechanics and physics
3. Sampling
   
4. Note: explain why, how, and why again.
   

Post-scan

1. Workflow:
   
   
   
   1. FOV alignment (2D, 3D),
      
   2. Polar coordinate system
      
   3. Segmentation: extracting tertiary glyphs: groove spacing area, groove valley, and groove bottom.
   4. Filtering: blobs from dirt and thresholding error from, e.g., damaged groove spacing.
   5. Restoring grooves: detaching, reattaching, sidewall removal, sting removal
      
   6. Groove edge extraction
      
   7. Groove unwrapping (windowing)
      
   8. Groove bottom thinning
      
   9. Resampling and fitting
      
   10. Stitching
       
   11. counter-EQ (optional)
       
   12. Stereo decoding and audio output.
       

Mechanics and Physics

1. VOS
   
2. Depth irregularity and Coarse-to-fine vertical scanning parameter derivation.
3. Resampling dillema.
4. Bottom capturing: Ramp intensity
5. Stage drift and counter-strategy.
   

tertiary build-up + sting removal bug fix

tertiary is needed twice.

1. 1. sting removal
2. 2. bottom thinning
3. so, need a function
4. [vaa_pair, ca_vaa_btm] = tertiary(vaa_in, vaa_btm, vaa_out, common_range)

Sting removal fix

1. previously we use inner-bottom, outer-bottom pair without pairing up inner/outer,
   
2. this is wrong.
3. first, removal angular redundancy,
   
4. then, built tertiary,
5. finally, remove sting via w2z verification.

small blobs percentage

2825/3223 = 87.65%

bottom 1d from 2d

Problem

1. radial thinning disregard neighboring azimuths and creates choppy line
2. tangential thinning tends to drift apart, i.e., deviate from the ideal center of groove bottom

Solution

1. somehow combine the two: weighted average of the two results.

Analysis

1. what we do NOT want to see: a locally(at its own azimuth) optimized btm-pos that disintegrates from its context.
   
1. 1D-btm(a) deviates to the back or front (tangential error): goes behind the previous point or beyond the next point.
2. 1D-btm(a) deviates to the left or right (radial error): causes w/z to fluctuate (stylus shape distorted).
   
2. what we want the 1D-btm(a) to provide: 
1. a(i) > a(i-1)
2. abs(r(i)-r(i-1))<e, e.g. 2pixels
   
3. dr_btm is similar to dr_in, dr_out
4. abs(w2z(i) - w2z(i-1)) < e

Imp

1. #1 elem: 1D-btm(1) is chosen according to any of : 1) lowest, 2) middle, 3) equal w/z for in/out.
2. #2+ elems:
   1. vp = closest_to_line(e_in(a), e_out(a))
   2. vpp = find(vp(a)>a(i-1))
   3. p = min(abs(r(a) - r(a-1))) or
      
   4. p = P * min( (w2z_out(a) - w2z_out(a-1)) / w2z_out(a-1) ) + Q * min(.... w2z_in....)
      
   
   

thresholding: cc analysis

200 * wavelength(82.5415nm) = 15um

This corresponds to our measured min-depth of grooves.

Previously, we used a histogram and try to find the population drop, based on the fact that bottom is thin and thus contains much less points than top. And tried to refine this value by applying a depth difference thresholding.

This method fails at noisy depth data and the changing depth of grooves, where bottom can be fairly close to some part of the top. And when top depths vary, they scatter into different histogram bins with huge population difference, making thresholding based on population very hard.

Failed methods and params

1. rg_ratio_population_drop: values does not matter much
2. delta_z: essential
3. z_peak: most population, actual peak, bin_thresh+1
4. thresh = mean/median (z_peak, z_btm)

The above data didn't prove robust.
thresh = z_most_popular - magic_thresh, i.e., 15um.

CC analysis: based on bad-data first (thresholding not robust)

Problem

1. thresholding not robust, a overly hi-thresh tends to bring top-edges to the ground -> bottom.

Solution

1. find bad data (Inf) first, what's inside of it are bottoms, the rest of the FOV are tops.

Imp

1. good data <= 1, bad <=0,
   
2. CC analysis, 
3. get bad-contained CCs -> bottom
   

detach: sidewall-faked bottom causes false-alarm

Analysis

1. If no holes in b/w the two btm-chains of a CC are found, mark this as non-attached CC
   

sting removal

Problem

1. sting removal tends to ruin non-attached/broken FOVs.
   

Solution

1. only apply the sting removal to attached/broken FOVs.
2. for other FOVs, only apply radial-redundancy removal.
   

bottom 1D from 2D wrong!

X: 1D bottom azimuth positions
Y: their index in original bottom 2D band

What's wrong here?

1. Some consecutive azimuths have same bottom 2D points (overlapped). They should have been monotonously increasing indices.
2. Instead, this curve means we will have "blur" problem if not angularly resampling/filtering carefully.
3. This also means, bottom thinning is problematic. Neither deepest-rule nor median-radius-rule would give us smooth continuous bottom trace.
   

How to solve this

1. resampling over each azimuth, if the tertiary has any overlap with the previous, it has to be discarded or adjusted to pair up other points.
2. we need to delay the fix? verify whether this fix will significantly improve sound quality?