|
|
Pages: 1 2 3 [4] 
|
 |
|
 Author
|
Topic: sources with Orcas (Read 5704 times)
|
|
Jason G
|
The order of the signals within the output result file never matters, but I can see no practical way to select the right subset of what may be a very large number of potential signals other than using the same sequence of searches as stock. Ahh I see, a sticky problem. Just musing some more, getting a better picture, statistically might there be a strong subset of overflow cases where the dataset tends to white noise?(I realise probably all the good data probably does anyway  ) And in such cases would the first 31 signals be definitely pulses?[or spikes rather] Prechecking for possible overflow is certainly an interesting concept. If someone came up with a really efficient way to do that, the project might consider putting that code in the splitter. In the science app, maybe the best opportunity is during baseline smoothing. Don't know about efficient  . I would, perhaps incorrectly, expect at least some types of obvious overflow signal [tasks] to be fairly 'white'. It's been a long time since I looked at an autocorrelation function, from vague memory they involve a single convolution. Something like that would be able to judge the whiteness of the signal against a decided threshold dirac figure/function. I've used them in signal processing many years ago for analysis of buried periodic signals. Subtracting the autocorrelation of white noise from that of the source [ had interesting results].... but that was on a 1k node torus so algorithmic complexity and other practical considerations weren't an issue under much consideration  . [though they should've been] I'll also note that if we found a way of dividing the work much more effectively, the changes could be applied to the official sources prior to the next stock release. That release could be named setiathome_multibeam or something similar, and all participants would have to upgrade.
Joe
That I'll leave for thought 'till next week when I'm on my holidays... yay... I haven't been following what Heinz is up to there, I was lost somewhere around 'Leeks' but you gave some food for thought and I'll figure it all out then. 'Till next week Jason |
|
|
|
« Last Edit: 11 Dec 2007, 03:24:25 am by j_groothu »
|
Logged
|
Current Repositories: (pw access only) svn://sinbadsvn.dyndns.org/boinc , (active) svn://sinbadsvn.dyndns.org/SaH_v6b/....[trunk/branch]... (active)
|
|
|
|
Josef W. Segur
|
... I would, perhaps incorrectly, expect at least some types of obvious overflow signal [tasks] to be fairly 'white'. It's been a long time since I looked at an autocorrelation function, from vague memory they involve a single convolution. Something like that would be able to judge the whiteness of the signal against a decided threshold dirac figure/function. I've used them in signal processing many years ago for analysis of buried periodic signals. Subtracting the autocorrelation of white noise from that of the source [ had interesting results].... but that was on a 1k node torus so algorithmic complexity and other practical considerations weren't an issue under much consideration . [though they should've been] ... Jason I've always thought of the WUs as consisting of a lot of white noise out of which we try to extract what isn't. Although the ALFA receivers are recent state of the art with 300 MHz. bandwidth, what goes on the Multibeam recorder is 2 bit complex values representing only the signs of the real and imaginary complex data points of the 2.5 MHz. bandwidth we're examining. That digitization represents a lot of noise, no? Then after the recording is received at Berkeley, the Splitter expands the data into complex single floats, does forward FFTs at 2048 length and inverse FFTs at length 8 and repacks the output in 2 bit form to be placed in the WUs. As a practical matter, most overflows have been on Spikes until the recent radar RFI problem caused a lot of Triplets. The record for my Willamette P4 shows 1682 Gaussians, 1807 Pulses, 11764 Spikes, and 10718 Triplets. 8162 of those Triplets are from one data set we crunched at SETI Beta before the radar RFI was understood. That's from 3524 results. The thresholds are nominally adjusted to give about an even chance of finding a signal or not if the noise level is normal, the Gaussian and Pulse counts match that but the excess Spike count probably reflects cases where random RFI caused overflows. Note that Spikes aren't exactly what the name implies, they are found by examining the power spectrum of a single FFT; if there's one frequency that's more than 24 times more powerful than the average it's counted as a Spike. A continuous narrow band interfering signal will cause a sequence of Spike reports as the FFTs are stepped through the duration of the data. Joe |
|
|
|
|
Logged
|
|
|
|
|
Jason G
|
I've always thought of the WUs as consisting of a lot of white noise out of which we try to extract what isn't. Although the ALFA receivers are recent state of the art with 300 MHz. bandwidth, what goes on the Multibeam recorder is 2 bit complex values representing only the signs of the real and imaginary complex data points of the 2.5 MHz. bandwidth we're examining. That digitization represents a lot of noise, no? Then after the recording is received at Berkeley, the Splitter expands the data into complex single floats, does forward FFTs at 2048 length and inverse FFTs at length 8 and repacks the output in 2 bit form to be placed in the WUs. Yes but no. Based on some obscure reading on the black art of antennae, If I interpret [crudely but] correctly [That's a pretty big if] I can think of two ways that method of complex representation can reduce noise in the digitisation process. 1) Dumping the magnitude data & relying instead on a temporal persistence summation- The magnitude data [if it ever existed] probably would amount to random noise like we've been discussing, I gather direct digitisation of magnitude in this realm would rely more heavily on the analog characteristics of the receiver. Also In the method that is used, from what I gather the signal polarity is very reliable in such a setup, and already digital, and the time characteristic inherently precise, so vector summation (If that's what the splitter process simplistically amounts to maybe) of those signals to form a single float signal, would be as precise time-wise. So the resynthesis should be a truer effective representation of the magnitude. 2) Complex representaion cheats Nyquist: As I understand it we now get 'n' overall bandwidth capability instead of 'n/2' before nyquist induced aliasing takes over. I think that's a pretty remarkable 'something for nothing'. As a practical matter, most overflows have been on Spikes until the recent radar RFI problem caused a lot of Triplets. The record for my Willamette P4 shows 1682 Gaussians, 1807 Pulses, 11764 Spikes, and 10718 Triplets. 8162 of those Triplets are from one data set we crunched at SETI Beta before the radar RFI was understood. That's from 3524 results. The thresholds are nominally adjusted to give about an even chance of finding a signal or not if the noise level is normal, the Gaussian and Pulse counts match that but the excess Spike count probably reflects cases where random RFI caused overflows. Note that Spikes aren't exactly what the name implies, they are found by examining the power spectrum of a single FFT; if there's one frequency that's more than 24 times more powerful than the average it's counted as a Spike. A continuous narrow band interfering signal will cause a sequence of Spike reports as the FFTs are stepped through the duration of the data.
Joe Thanks, that description of spikes helps greatly trying to get a clearer picture of the processing sequence, I'll try to mentally process the implications. Given the definition of a spike, and that it is the predominant indicator of noise, It may be that the only feasible method of determining a noisy workunit is indeed to process it to completion (Excepting, of course, known environmental influences, like we can make use of the radar blanking)... We are the 'precheckers'... Jason |
|
|
|
|
Logged
|
Current Repositories: (pw access only) svn://sinbadsvn.dyndns.org/boinc , (active) svn://sinbadsvn.dyndns.org/SaH_v6b/....[trunk/branch]... (active)
|
|
|
|
_heinz
|
Hi all, my evaluation installation of Orcas is now expired. I installed the new VS2008 Express Edition yesterday. I can open and work with all projectfiles from Orcas without any problems, parallel to VS2005 Express Edition. Thats a really good news. have fun heinz |
|
|
|
|
Logged
|
|
|
|
|
|
|
|
|
Pages: 1 2 3 [4] 
|
|
|
|
Quote!
Every solution breeds new problems.- Murphy's Law
|
 |  |  |
| |
| Site Statistics |
| Total Members: | 909 |
| Total Posts: | 7,741 |
| Total Topics: | 396 | | Downloads |
| Apps |
| Windows R-1.x | 24,923 |
| Windows R-2.0 | 20,165 |
| Windows R-2.2 | 35,933 |
| Linux 32bit 1.x | 6,448 |
| Linux 32bit 2.2 | 4,120 |
| Linux 64bit 2.2 | 1,598 |
| Alpha/IA64 | 154 |
| FreeBSD | 498 |
| HPUX | 291 |
| Subtotal: | 93,187 |
| Source packs: | 3,822 |
| Tool/WU packs: | 7,151 |
| Total: | 136,675 | | GBs dl'd: | 272.95 | | Pages served |
| Today: | 1,393 |
| Total: | 2,755,398 |
| (since 6/26/2006) |
| 173 Donations to S@H |
| U.S. Dollars: | 3,196.59 |
| Euros: | 863.90 |
| Last 24h: | $ 0.00 |
| Avg./24h: | $ 9.41 |
| Estim. total: | $ 4,319.66 |
Latest Member:
From Zero Land |
| |
 | |  |
| | |
| |
Online users/last 15m
16 Guests, 2 Users
Jason G, Hans Dorn
33 Members/last 24hJason G, Hans Dorn, Urs Echternacht, Devaster, Vyper, ahaveland, KWSN - jonnyv, zvonas, Marker, petefan, msattler, Gecko_R7, Yin Gang, popandbob, WinterKnight, FalconFly, Fivestar Crashtest, Fredericx51, Josef W. Segur, hiamps, Gus, Mr.Microwave, sunu, Raistmer, From Zero Land, Yurik, Geek@Play, JDWhale, UncleVom, KarVi, ajs, The Grinch, Crunch3r
| |
| | |
|
