| Age | Commit message (Collapse) | Author |
|
|
|
- add weepinbell.http.client/with-retry macro
- retry only requests that returns :error with type TimeoutException
- retry is done asynchronously
- with-retry should return a Future
- retry verifier requesting source and target, 3 times with 5 seconds
backoff
|
|
- some namespaces tests require did not get renamed
- url/extract might throw AssertionError if input contains non-HTTP(S)
urls. it should not check schemes.
- verifier doesn't need to filter non-HTTP(S) urls since they will
return false anyway, and plus it makes localhost urls in tests fail
|
|
add docstrings
format code
|
|
- URISyntaxException was not caught in url/normalize, causing
normalization failure which leads to verification failure when there is
one invalid URL, even though valid URL exists
- skip non-HTTP(S) urls when verifying
|
|
utils: add url utilities for accurate comparison
- add url/normalize taht reconstructs URL, strips unnecessary
components, reorders query string and ensures determinism
- add url/url= that uses url/normalize
receiver: normalize URLs before checking and putting in queue
- use url= and normalize from utils.url
- rename verify-request to check-request to avoid confusion with
verifier.clj
verifier: check if source includes target URL more accurately
- implement url/extract with robinst/autolink-java, which extracts a
vector a URLs from string
- check if source includes target by
1. extracting all links from source body
2. normalizing all links before comparing with target link (with url=
func)
- refactor verify-headers -> match-content-types, with a higher order
function and clearer description
retriever, storage: use url= instead of = when comparing urls
|
|
- replace mf2j with mf2clj
- remove utils.clj completely
- refactored processor, use Chain of Responsibilities pattern and pass
around maps to reuse data (so we don't have to make a separate HTTP
request just to parse microformats, etc.)
- test: add more test cases to verifier test
- test: add basic test cases to all processing steps
- add more logging to each processing steps. good for debugging.
|