This adds a middleware for overload protection that is intended to help protect against malicious scrapers.
It does this via [`codel`](https://github.com/bohde/codel), which will perform the following:
1. Limit the number of in-flight requests to some user-defined max
2. When in-flight requests have reached their begin queuing requests.
Logged-in requests having priority above logged-out requests
3. Once a request has been queued for too long,
it has a probabilistic chance to be rejected based on how overloaded the entire system is.
When a server experiences more traffic than it can handle,
this keeps latency low for logged-in users and rejects just
enough requests from logged-out users to not overload the service.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
Fixes#33544
Adds two new api endpoints to list a versions of a package and to get
the latest version of a package by API.
⚠️ BREAKING ⚠️
the `size` field for this endpoint changes from `Size` to `size`.
This adds a cache for common package queries in `GetPackageDescriptor`.
Code which needs to process a list of packages benefits from this
change. This skips 350 queries in the package integration tests for
example.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This PR moved git attributes related code to `modules/git/attribute` sub
package and moved language stats related code to
`modules/git/languagestats` sub package to make it easier to maintain.
And it also introduced a performance improvement which use the `git
check-attr --source` which can be run in a bare git repository so that
we don't need to create a git index file. The new parameter need a git
version >= 2.40 . If git version less than 2.40, it will fall back to
previous implementation.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: yp05327 <576951401@qq.com>
Fix#2616
This PR adds a new sort option for exclusive labels.
For exclusive labels, a new property is exposed called "order", while in
the UI options are populated automatically in the `Sort` column (see
screenshot below) for each exclusive label scope.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
When list commits, some of the commits authors are the same at many
situations. But current logic will always fetch the same GPG keys from
database. This PR will cache the GPG keys, emails and users for the
context so that reducing the database queries.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This PR uniform all temporary directory usage so that it will be easier
to manage.
Relate to #31792
- [x] Added a new setting to allow users to configure the global
temporary directory.
- [x] Move all temporary files and directories to be placed under
os.Temp()/gitea.
- [x] `setting.Repository.Local.LocalCopyPath` now will be
`setting.TempPath/local-repo` and the customized path is removed.
```diff
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;[repository.local]
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;;
-;; Path for local repository copy. Defaults to TEMP_PATH + `local-repo`, this is deprecated and cannot be changed
-;LOCAL_COPY_PATH = local-repo
```
- [x] `setting.Repository.Upload.TempPath` now will be
`settting.TempPath/uploads` and the customized path is removed.
```diff
;[repository.upload]
-;;
-;; Path for uploads. Defaults to TEMP_PATH + `uploads`
-;TEMP_PATH = uploads
```
- [x] `setting.Packages.ChunkedUploadPath` now will be
`settting.TempPath/package-upload` and the customized path is removed.
```diff
;[packages]
-;;
-;; Path for chunked uploads. Defaults it's `package-upload` under `TEMP_PATH` unless it's an absolute path.
-;CHUNKED_UPLOAD_PATH = package-upload
```
- [x] `setting.SSH.KeyTestPath` now will be
`settting.TempPath/ssh_key_test` and the customized path is removed.
```diff
[server]
-;;
-;; Directory to create temporary files in when testing public keys using ssh-keygen,
-;; default is the system temporary directory.
-;SSH_KEY_TEST_PATH =
```
TODO:
- [ ] setting.PprofDataPath haven't been changed because it may need to
be kept until somebody read it but temp path may be clean up any time.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Fix#28144
To make the resources will be cleanup once failed. All repository
operations now follow a consistent pattern:
- 1. Create a database record for the repository with the status
being_migrated.
- 2. Register a deferred cleanup function to delete the repository and
its related data if the operation fails.
- 3. Perform the actual Git and database operations step by step.
- 4. Upon successful completion, update the repository’s status to
ready.
The adopt operation is a special case — if it fails, the repository on
disk should not be deleted.
Extract from #33934
In the same goroutine, we should reuse the exist cat file sub process
which exist in `git.Repository` to avoid creating a unnecessary
temporary subprocess.
This PR reuse the exist cate file writer and reader in
`getCommitFromBatchReader`.
It also move `prepareLatestCommitInfo` before creating dataRc which will
hold the writer so other git operation will create a temporary cat file
subprocess.
* Fix#33972
* Use consistent path resolving for links and medias.
* No need to make the markup renders to resolve the paths, instead, the
paths are all correctly resolved in the "post process" step.
* Fix#33274
* Since 1.23, all paths starting with "/" are relative to current render
context (for example: the current repo branch)
* Introduce `/:root/path-relative-to-root`, then the path will be
rendered as relative to "ROOT_URL"
Fix#32886
Add `last_committer_date` and `last_author_date` in the content API
which is not implemented by Github API v3 at the moment.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Adds the `flat-square` style to action badges. Styles can be selected by
adding `?style=<style>` to the badge endpoint. If no style query is
given, or if the query is invalid, the style defaults to `flat`.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
1. Make the material icon falls back to basic theme correctly
2. Remove `TestAttributeReader`, the problem has been resolved.
3. Fix `toggleElem` bug and add tests
Fix#33966
```
;; User must sign in to view anything.
;; It could be set to "expensive" to block anonymous users accessing some pages which consume a lot of resources,
;; for example: block anonymous AI crawlers from accessing repo code pages.
;; The "expensive" mode is experimental and subject to change.
;REQUIRE_SIGNIN_VIEW = false
```
The current action status badge are looking different from most other
badges renders, which is especially noticeable when using them along
with other badges. This PR updates the action badges to match the
commonly used badges from other providers.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Close#34022 , #33550
This error message always appears when using the `check-attr` command,
even though it works correctly.
The issue occurs when the stdin writer is closed, so I added a special
case to handle and check the error message when the exit code is 1.
The old logic is incomplete. See the comment for the improved logic.
Fix#34011
And more fixes:
1. use empty "alt" for images, otherwise the width is not right when the
image fails to load
2. remove the "dropdown icon" patch, because it has been clearly done in
"dropdown.js" now
3. remove the "dropdown filtered item" patch, added a clear callback,
and improve the logic
4. fix global init when a node is removed and added back gain (eg: the
"cherry pick" dialog with a dropdown)
1. Ignore empty inputs in `UnmarshalHandleDoubleEncode`
2. Ignore non-existing `stateEvent.User` in gitlab migration
3. Enable `release` and `wiki` units when they are selected in migration
4. Sanitize repo name for migration and new repo
The pagination on the user dashboard sounds unnecessary, this will
change it to a prev/next buttons. For instances with around `10 million`
records in the action table, this option affects how the user dashboard
is loaded on first visit.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
This change targets https://github.com/go-gitea/gitea/issues/32663
We drop the hardcoded timeout of 60 seconds for requests to the internal
hook api. With this change the timeout is completly removed.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Resolve#29328
This pull request introduces a file tree on the left side when reviewing
files of a repository.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Make `SearchMode` have default value if it is empty
* Add some comments for the "match" queries
* Fix a copy-paste mistake in `buildMatchQuery` (`db.go`)
* Add missing `q.Analyzer = repoIndexerAnalyzer`, it is in old code,
although I do not see real difference ....
