Basic functions and configuration stuff

Implements:

  • Cache
  • Config
  • MJD
  • UTC, UTCnow

class Cache[source]

Cache(path, clear:bool=False) :: dict

Manage a file cache

  • path -- string or filepath object
    This is the folder where the index and data files are saved.
  • clear -- set True to clear the cache on initialization

This uses pickle to save objects, associated with a hashable key, which is used to index the filename in a file index.pkl in the same folder.

The __call__ function is a convenient way to use it, so one call may either store a new entry or retrieve an existing one.

Cache.__call__[source]

Cache.__call__(key, func, *pars, description='', overwrite=False, **kwargs)

One-line usage interface for cache use

  • key -- key to use, usually a string. Must be hashable
    If None, ignore cache and return the function evaluation
  • func -- user function that will return an object that can be pickled
  • pars, kwargs -- pass to func
  • description -- optional string that will be printed
  • overwrite -- if set, overwrite previous entry if exists

Example:

mycache = Cache('/tmp/thecache', clear=True)

def myfun(x):
    return x

result = mycache('mykey', myfun, x=99,  description='My data')

Cache test

def cache_test(path):
    c = Cache(path, clear=True)

    # simmple interface
    c.add('one', 'one');
    c.add('two', 'two')
    c.add('two', 'two') # getnerates warning
    if path is not None:
        assert c.get('two') == 'two'

    # test function interface
    func = lambda x:f'value: {x}'
    
    r1 = c('four',  func,  4, description='Test')
    r2 = c('four',  func,  5,  description='Test') #should not get called
    assert c.path is None or r1==r2, f'{r1}, {r2}'
    
    # remaving an entry
    print(f'Before remove:\n{c}')
    assert 'four' in c
    c.remove('four')
    assert 'four' not in c
    print(f'After remove:\n{c}')
    c.clear()

test_path = Path('/tmp/cache_test')
test_path.mkdir(exist_ok=True)
cache_test(test_path)
Clearing cache!
Test: Saving to cache with key "four"
Test: Restoring from cache with key "four"
Before remove:
Cache contents
 key                                    size  time                 name, folder /tmp/cache_test
  one                                     18  2022-07-16 04:47     cache_file_a914b484ec77559.pkl
  two                                     18  2022-07-16 04:47     cache_file_252f4e04c4c9e39.pkl
  four                                    23  2022-07-16 04:47     cache_file_6306a9b495235f4.pkl

After remove:
Cache contents
 key                                    size  time                 name, folder /tmp/cache_test
  one                                     18  2022-07-16 04:47     cache_file_a914b484ec77559.pkl
  two                                     18  2022-07-16 04:47     cache_file_252f4e04c4c9e39.pkl

Warning: cached object for key "two" exists

class Config[source]

Config(**kwargs)

@ipynb_doc
def config_summary():
    """
    
    #### config.Config -- parameters are from three sources:
    - defaults
    - the file `~/.config/wtlike/config.yaml` if it exists
    - keyword args in Config constructor. For example, to suppress all printout:
        ```
    config = Config(verbose=0)
    ```
    
    ##### Config defaults
    This is yaml-format, corresponding to `config.yaml`.
    {config_defaults}
    
    
    #### Config contents as set up here
    {config_text}   


    
    ##### config.cache -- a file cache
    The class `Cache`, available from `config.cache` implements a file cache in the folder
    `{config.cachepath}`
    
    {cache_text}
    """
    config = Config()
    config_defaults = Config.defaults
    config_text = monospace(config, summary='config parameter list')

    try:
        cache_text = monospace(config.cache, 'current cache contents' )
    except Exception as msg:
        cache_text = f'(Failed: {msg})'


    return locals()
config_summary() 

config.Config -- parameters are from three sources:

  • defaults
  • the file ~/.config/wtlike/config.yaml if it exists
  • keyword args in Config constructor. For example, to suppress all printout:
    config = Config(verbose=0)
Config defaults

This is yaml-format, corresponding to config.yaml.

    verbose         : 1 # set to zero for no output
    warnings        : ignore # 

    datapath        : ~/wtlike-data # where to find data--must be set
    cachepath       : ~/.cache/wtlike #

    # Expect 4FGL FITS file, e.g.,  gll_psc_v28.fit
    catalog_file    : 

    # multiprocessing
    pool_size       : 1 # number of pool processes to use

    # data cuts, processing
    radius          : 4
    cos_theta_max   : 0.4
    z_max           : 100
    offset_size     : 2.e-06  # scale factor used for event time

    # binning -- actually determined by weight run
    energy_edge_pars : [2,6,17] # pars for np.logspace
    etypes          : [0, 1] # front, back
    nside           : 1024
    nest            : True

    # data selection for cell creation
    week_range      : []  # default all weeks found
    time_bins       : [0, 0, 7] # full MJD range, 7-day cells
    exp_min         : 5    # threshold for exposure per day, in cm^2 Ms units.

    # cell fitting
    use_kerr        : True  # Use the Kerr power-law exposure weighting
    likelihood_rep  : poisson
    poisson_tolerance : 0.2




Config contents as set up here

config parameter list
Configuration parameters 
verbose : 1
warnings : None
datapath : /mnt/c/Users/thbur/onedrive/fermi/wtlike-data
cachepath : /home/burnett/.cache/wtlike
catalog_file : /mnt/c/Users/thbur/onedrive/fermi/wtlike-data/gll_psc_v28.fit
pool_size : 4
radius : 4
cos_theta_max : 0.4
z_max : 100
offset_size : 2e-06
energy_edge_pars : [2, 6, 17]
etypes : [0, 1]
nside : 1024
nest : True
week_range : (None, None)
time_bins : [0, 0, 7]
exp_min : 5
use_kerr : True
likelihood_rep : poisson
poisson_tolerance : 0.2
keep_pixels : True
energy_edges : [1.00000000e+02 1.77827941e+02 3.16227766e+02 5.62341325e+02
1.00000000e+03 1.77827941e+03 3.16227766e+03 5.62341325e+03
1.00000000e+04 1.77827941e+04 3.16227766e+04 5.62341325e+04
1.00000000e+05 1.77827941e+05 3.16227766e+05 5.62341325e+05
1.00000000e+06]
energy_bins : [1.33352143e+02 2.37137371e+02 4.21696503e+02 7.49894209e+02
1.33352143e+03 2.37137371e+03 4.21696503e+03 7.49894209e+03
1.33352143e+04 2.37137371e+04 4.21696503e+04 7.49894209e+04
1.33352143e+05 2.37137371e+05 4.21696503e+05 7.49894209e+05]
error_msg :

config.cache -- a file cache

The class Cache, available from config.cache implements a file cache in the folder /home/burnett/.cache/wtlike

current cache contents
Cache contents
key size time name, folder /home/burnett/.cache/wtlike
PSR J0835-4510_data 156120367 2022-06-06 12:55 cache_file_ae6854be273acdc.pkl
PSR J0633+1746_data 115577595 2022-06-06 13:00 cache_file_5707380cef0a5b.pkl
PSR J1709-4429_data 120303131 2022-06-06 13:05 cache_file_7500f4770cd889.pkl
PSR J2021+4026_data 130940729 2022-06-06 13:11 cache_file_a8dc38f04aab018.pkl
PSR J1836+5925_data 125537569 2022-06-06 13:17 cache_file_a2b8a18b80eb164.pkl
PSR J2021+3651_data 126539471 2022-06-06 13:23 cache_file_6661649ffbd07a9e.pkl
PSR J0007+7303_data 134394303 2022-07-04 09:49 cache_file_cfa096edb809c60.pkl
PSR J1809-2332_data 116347883 2022-06-06 13:40 cache_file_fb3fdcb2262565.pkl
PSR J1826-1256_data 122240951 2022-06-06 13:45 cache_file_335ec043a4024484.pkl
PSR J1907+0602_data 118468339 2022-06-06 13:51 cache_file_55b744805b5c9ff9.pkl
PSR J1418-6058_data 132775053 2022-06-06 13:58 cache_file_29d5c1a8500b93e8.pkl
PSR J1057-5226_data 101444077 2022-06-06 14:03 cache_file_384e41fd4ebdac81.pkl
PSR J1813-1246_data 117386407 2022-06-06 14:08 cache_file_08c2148395b6f1f.pkl
PSR J1028-5819_data 122583649 2022-06-06 14:15 cache_file_812ad9ec583a961.pkl
PSR J2229+6114_data 133820383 2022-06-06 14:21 cache_file_e476bf55a06fa3e.pkl
PSR J1048-5832_data 120018647 2022-06-06 14:27 cache_file_70b0714d172f3f2.pkl
PSR J1732-3131_data 123636019 2022-06-06 14:33 cache_file_19edcba9cfff048.pkl
PSR J1413-6205_data 132165683 2022-06-06 14:40 cache_file_6b3b60386724b1db.pkl
PSR J1747-2958_data 126607289 2022-06-06 14:46 cache_file_2f67d22cb93688bb.pkl
PSR J1952+3252_data 114920081 2022-06-06 14:52 cache_file_512aa5183afaad0.pkl
P88Y3243_data 88630555 2022-06-07 05:16 cache_file_66a9d9663efb97.pkl
P88Y5020_data 119817913 2022-06-07 10:33 cache_file_29e7a2e2944fb978.pkl
504H-0317_data 130475249 2022-06-07 10:39 cache_file_cb1897be30807d5.pkl
P88Y6266_data 99892151 2022-06-13 17:57 cache_file_35907c8f78cbed7c.pkl
P88Y2172_data 117721945 2022-06-17 11:49 cache_file_175f18c1a956e4d.pkl
P88Y0096_data 133548253 2022-07-04 09:54 cache_file_3c73e1e34f4c55.pkl
P88Y0521_data 95747295 2022-07-02 09:46 cache_file_7f6ab463d3ea18fd.pkl
P88Y4060_data 99826151 2022-07-02 10:05 cache_file_43042fb37426e122.pkl
P88Y1134_data 94193191 2022-07-02 09:51 cache_file_24a8a2282c30fdea.pkl
P88Y6057_data 89569443 2022-07-02 10:10 cache_file_4c75eeb98fde1be8.pkl
P88Y1710_data 125661037 2022-07-02 09:56 cache_file_342f3a94f9ee1f36.pkl
P88Y1932_data 87756585 2022-07-02 10:00 cache_file_5d597895910d1291.pkl
P88Y0643_data 97581865 2022-07-03 08:42 cache_file_4f9f229b3e50c0d6.pkl
P88Y0514_data 96424943 2022-07-03 13:01 cache_file_49fb71b2c22b562d.pkl
PSR J0633+1746_weeks_9-11 419065 2022-07-04 10:03 cache_file_dd5ac11dbd01293.pkl
P88Y3157_weeks_9-165 20704939 2022-07-04 10:06 cache_file_cd697fdc73ce09.pkl
PSR J0534+2200_data 111891877 2022-07-13 01:08 cache_file_737398512a45c22.pkl
P88Y3157_data 88180753 2022-07-13 01:15 cache_file_5f7b91e889cf5dcf.pkl
P88Y0615_data 94004743 2022-07-14 13:03 cache_file_19435ee6bf1a46a2.pkl
PSR J2021+4026-nk 138951485 2022-07-15 09:24 cache_file_6e0c2e81f58fab52.pkl

Time conversion

  • MET: mission elapsed time
  • MJD: modified Julian date (days)
  • UTC: ISO time
  • UTCnow: current ISO time

MJD[source]

MJD(arg)

convert MET or UTC to MJD

UTC[source]

UTC(mjd)

convert MJD value to ISO date string

UTCnow[source]

UTCnow()

current UTC

mission_week[source]

mission_week(mjd)

return the mission week number for a MJD value (Note that week #0 starts on UTC Thursday 2008-05-29 00:00, the first data is in week 9, and that week 525 is missing)

mjd_range[source]

mjd_range(start, stop, make_round=True)

Return a tuple with a valid MJD range If either is greater than first_data, interpret as MJD otherwise offset from first_data, or now

So 0,0 is full range, -7, 0 is last 7 days, 0,7 is first 7 days

UTC(MJD(0)), UTC(first_data), UTCnow(), MJD('now'), UTC(MJD('now')), mjd_range(0,0)
('2001-01-01 00:01',
 '2008-08-05 00:00',
 'UTC 2022-07-16 11:47',
 59776.49153800692,
 '2022-07-16 11:47',
 (54683, 59776))
np.arange(*[mission_week(x) for x in mjd_range(0,100)])
array([ 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23])
assert UTC(MJD(0))=='2001-01-01 00:01'
assert MJD('2008')==54466

class FermiInterval[source]

FermiInterval(interval=30, offset=0)

For iteration thru (start,stop) tuples in the Fermi data set

class FermiMonth[source]

FermiMonth() :: FermiInterval

For iteration thru (start,stop) tuples in the Fermi data set

class FermiWeek[source]

FermiWeek() :: FermiInterval

For iteration thru (start,stop) tuples in the Fermi data set

Miscellaneous utilities

  • bins_size_name
  • decorate_with
  • Timer

bin_size_name[source]

bin_size_name(bins)

Provide a nice name, e.g., 'day' for a time interval

print([bin_size_name(x) for x in (1/96, 1,2,7, 14, 365.25)])
['15-min', '1-day', '2-day', '1-week', '2-week', '1-year']

decorate_with[source]

decorate_with(other_func)

class Timer[source]

Timer()

Usage:

with Timer() as t:
    time.sleep(5)
print(t)
with Timer() as t:

    time.sleep(2)
    print('intemediate',t)
    time.sleep(3)
print('Final',t)
assert(abs(t.elapsed -5)<0.1), f'wrong elapsed time: {t.elapsed}'
intemediate elapsed time: 2.0s (0.0 min)
Final elapsed time: 5.0s (0.1 min)