summaryrefslogtreecommitdiff
path: root/scripts/lib/kdoc/kdoc_parser.py
AgeCommit message (Collapse)Author
2025-08-18docs: kdoc: remove redundant comment strippingJonathan Corbet
By the time stuff gets to create_parameter_list(), comments have long since been stripped out, so we do not need to do it again here. Acked-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250814154035.328769-8-corbet@lwn.net
2025-08-18docs: kdoc: tighten up the pointer-to-function caseJonathan Corbet
Tighten up the code and remove an unneeded regex operation. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250814154035.328769-7-corbet@lwn.net
2025-08-18docs: kdoc: tighten up the array-of-pointers caseJonathan Corbet
Simplify one gnarly regex and remove another altogether; add a comment describing what is going on. There will be no #-substituted commas in this case, so don't bother trying to put them back. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250814154035.328769-6-corbet@lwn.net
2025-08-18docs: kdoc: add a couple more comments in create_parameter_list()Jonathan Corbet
Make what the final code is doing a bit more clear to slow readers like me. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250814154035.328769-5-corbet@lwn.net
2025-08-18docs: kdoc: clean up the create_parameter_list() "first arg" logicJonathan Corbet
The logic for finding the name of the first in a series of variable names is somewhat convoluted and, in the use of .extend(), actively buggy. Document what is happening and simplify the logic. Acked-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250814154035.328769-4-corbet@lwn.net
2025-08-18docs: kdoc: tidy up space removal in create_parameter_list()Jonathan Corbet
Remove a redundant test and add a comment describing what the space removal is doing. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250814154035.328769-3-corbet@lwn.net
2025-08-18docs: kdoc: remove dead codeJonathan Corbet
create_parameter_list() tests an argument against the same regex twice, in two different locations; remove the pointless extra tests and the never-executed error cases that go with them. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250814154035.328769-2-corbet@lwn.net
2025-08-11Merge branch 'dump-struct' into docs-mwJonathan Corbet
In my ongoing effort to truly understand our new kernel-doc, I continue to make changes to improve the code, and to try to make the understanding task easier for the next person. These patches focus on dump_struct() in particular, which starts out at nearly 300 lines long - to much to fit into my little brain anyway. Hopefully the result is easier to manage. There are no changes in the rendered docs.
2025-08-11docs: kdoc: a few final dump_struct() touchesJonathan Corbet
Add a couple more comments so that each phase of the process is now clearly marked. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-13-corbet@lwn.net
2025-08-11docs: kdoc: extract output formatting from dump_struct()Jonathan Corbet
The last thing done in dump_struct() is to format the structure for printing. That, too, is a separate activity; split it out into its own function. dump_struct() now fits in a single, full-hight editor screen. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-12-corbet@lwn.net
2025-08-11docs: kdoc: further rewrite_struct_members() cleanupJonathan Corbet
Get rid of some redundant checks, and generally tighten up the code; no logical change. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-11-corbet@lwn.net
2025-08-11docs: kdoc: Some rewrite_struct_members() commentingJonathan Corbet
Add comments to rewrite_struct_members() describing what it is actually doing, and reformat/comment the main struct_members regex so that it is (more) comprehensible to humans. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-10-corbet@lwn.net
2025-08-11docs: kdoc: remove an extraneous strip() callJonathan Corbet
...the variable in question was already strip()ed at the top of the loop. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-9-corbet@lwn.net
2025-08-11docs: kdoc: rework the rewrite_struct_members() main loopJonathan Corbet
Adopt a more Pythonic form for the main loop of this function, getting rid of the "while True:" construction and making the actual loop invariant explicit. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-8-corbet@lwn.net
2025-08-11docs: kdoc: split struct-member rewriting out of dump_struct()Jonathan Corbet
The massive loop that massages struct members shares no data with the rest of dump_struct(); split it out into its own function. Code movement only, no other changes. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-7-corbet@lwn.net
2025-08-11docs: kdoc: split top-level prototype parsing out of dump_struct()Jonathan Corbet
Move the initial split of the prototype into its own function in the ongoing effort to cut dump_struct() down to size. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-6-corbet@lwn.net
2025-08-11docs: kdoc: move the prefix transforms out of dump_struct()Jonathan Corbet
dump_struct is one of the longest functions in the kdoc_parser class, making it hard to read and reason about. Move the definition of the prefix transformations out of the function, join them with the definition of "attribute" (which was defined at the top of the file but only used here), and reformat the code slightly for shorter line widths. Just code movement in the end. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-5-corbet@lwn.net
2025-08-11docs: kdoc: backslashectomy in kdoc_parserJonathan Corbet
A lot of the regular expressions in this file have extraneous backslashes that may have been needed in Perl, but aren't helpful here. Take them out to reduce slightly the visual noise. Escaping of (){}[] has been left in place, even when unnecessary, for visual clarity. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-4-corbet@lwn.net
2025-08-11docs: kdoc: Move a regex line in dump_struct()Jonathan Corbet
The complex struct_members regex was defined far from its use; bring the two together. Remove some extraneous backslashes while making the move. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-3-corbet@lwn.net
2025-08-11docs: kdoc: consolidate the stripping of private struct/union membersJonathan Corbet
There were two locations duplicating the logic of stripping private members and associated comments; coalesce them into one, and add some comments describing what's going on. Output change: we now no longer add extraneous white space around macro definitions. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-2-corbet@lwn.net
2025-07-31Merge tag 'docs-6.17' of git://git.lwn.net/linuxLinus Torvalds
Pull documentation updates from Jonathan Corbet: "It has been a relatively busy cycle for docs, especially the build system: - The Perl kernel-doc script was added to 2.3.52pre1 just after the turn of the millennium. Over the following 25 years, it accumulated a vast amount of cruft, all in a language few people want to deal with anymore. Mauro's Python replacement in 6.16 faithfully reproduced all of the cruft in the hope of avoiding regressions. Now that we have a more reasonable code base, though, we can work on cleaning it up; many of the changes this time around are toward that end. - A reorganization of the ext4 docs into the usual TOC format. - Various Chinese translations and updates. - A new script from Mauro to help with docs-build testing. - A new document for linked lists - A sweep through MAINTAINERS fixing broken GitHub git:// repository links. ...and lots of fixes and updates" * tag 'docs-6.17' of git://git.lwn.net/linux: (147 commits) scripts: add origin commit identification based on specific patterns sphinx: kernel_abi: fix performance regression with O=<dir> Documentation: core-api: entry: Replace deprecated KVM entry/exit functions docs: fault-injection: drop reference to md-faulty docs: document linked lists scripts: kdoc: make it backward-compatible with Python 3.7 docs: kernel-doc: emit warnings for ancient versions of Python Documentation/rtla: Describe exit status Documentation/rtla: Add include common_appendix.rst docs: kernel: Clarify printk_ratelimit_burst reset behavior Documentation: ioctl-number: Don't repeat macro names Documentation: ioctl-number: Shorten macros table Documentation: ioctl-number: Correct full path to papr-physical-attestation.h Documentation: ioctl-number: Extend "Include File" column width Documentation: ioctl-number: Fix linuxppc-dev mailto link overlayfs.rst: fix typos docs: kdoc: emit a warning for ancient versions of Python docs: kdoc: clean up check_sections() docs: kdoc: directly access the always-there KdocItem fields docs: kdoc: straighten up dump_declaration() ...
2025-07-17scripts: kdoc: make it backward-compatible with Python 3.7Mauro Carvalho Chehab
There was a change at kdoc that ended breaking compatibility with Python 3.7: str.removesuffix() was introduced on version 3.9. Restore backward compatibility. Reported-by: Akira Yokosawa <akiyks@gmail.com> Closes: https://lore.kernel.org/linux-doc/57be9f77-9a94-4cde-aacb-184cae111506@gmail.com/ Fixes: 27ad33b6b349 ("kernel-doc: Fix symbol matching for dropped suffixes") Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/d13058d285838ac2bc04c492e60531c013a8a919.1752218291.git.mchehab+huawei@kernel.org
2025-07-15docs: kdoc: emit a warning for ancient versions of PythonJonathan Corbet
Versions of Python prior to 3.7 do not guarantee to remember the insertion order of dicts; since kernel-doc depends on that guarantee, running with such older versions could result in output with reordered sections. Python 3.9 is the minimum for the kernel as a whole, so this should not be a problem, but put in a warning just in case somebody tries to use something older. Suggested-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: clean up check_sections()Jonathan Corbet
entry.sectcheck is just a duplicate of our list of sections that is only passed to check_sections(); its main purpose seems to be to avoid checking the special named sections. Rework check_sections() to not use that field (which is then deleted), tocheck for the known sections directly, and tighten up the logic in general. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: straighten up dump_declaration()Jonathan Corbet
Get rid of the excess "return" statements in dump_declaration(), along with a line of never-executed dead code. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: Regularize the use of the declaration nameJonathan Corbet
Each declaration type passes through the name in a unique field of the "args" blob - even though we have always just passed the name separately. Get rid of all the weird names and just use the common version. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: Coalesce parameter-list handlingJonathan Corbet
Callers to output_declaration() always pass the parameter information from self.entry; remove all of the boilerplate arguments and just get at that information directly. Formalize its placement in the KdocItem class. It would be nice to get rid of parameterlist as well, but that has the effect of reordering the output of function parameters and struct fields to match the order in the kerneldoc comment rather than in the declaration. One could argue about which is more correct, but the ordering has been left unchanged for now. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: use self.entry.parameterlist directly in check_sections()Jonathan Corbet
Callers of check_sections() join parameterlist into a single string, which is then immediately split back into the original list. Rather than do all that, just use parameterlist directly in check_sections(). Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: remove the "struct_actual" machineryJonathan Corbet
The code goes out of its way to create a special list of parameters in entry.struct_actual that is just like entry.parameterlist, but with extra junk. The only use of that information, in check_sections(), promptly strips all the extra junk back out. Drop all that extra work and just use parameterlist. No output changes. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: Centralize handling of the item section listJonathan Corbet
The section list always comes directly from the under-construction entry and is used uniformly. Formalize section handling in the KdocItem class, and have output_declaration() load the sections directly from the entry, eliminating a lot of duplicated, verbose parameters. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: drop "sectionlist"Jonathan Corbet
Python dicts (as of 3.7) are guaranteed to remember the insertion order of items, so we do not need a separate list for that purpose. Drop the per-entry sectionlist variable and just rely on native dict ordering. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-08scripts/kernel_doc.py: properly handle VIRTIO_DECLARE_FEATURESPaolo Abeni
The mentioned macro introduce by the next patch will foul kdoc; fully expand the mentioned macro to avoid the issue. Signed-off-by: Paolo Abeni <pabeni@redhat.com>
2025-07-02docs: kdoc: simplify the output-item passingJonathan Corbet
Since our output items contain their name, we don't need to pass it separately. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-02docs: kdoc; Add a rudimentary class to represent output itemsJonathan Corbet
This class is intended to replace the unstructured dict used to accumulate an entry to pass to an output module. For now, it remains unstructured, but it works well enough that the output classes don't notice the difference. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-01docs: kdoc: pretty up dump_enum()Jonathan Corbet
Add some comments to dump_enum to help the next person who has to figure out what it is actually doing. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-01docs: kdoc: some tweaks to process_proto_function()Jonathan Corbet
Add a set of comments to process_proto_function and reorganize the logic slightly; no functional change. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-01docs: kdoc: rework type prototype parsingJonathan Corbet
process_proto_type() is using a complex regex and a "while True" loop to split a declaration into chunks and, in the end, count brackets. Switch to using a simpler regex to just do the split directly, and handle each chunk as it comes. The result is, IMO, easier to understand and reason about. The old algorithm would occasionally elide the space between function parameters; see struct rng_alg->generate(), foe example. The only output difference is to not elide that space, which is more correct. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-30docs: kdoc: remove the brcount floor in process_proto_type()Jonathan Corbet
Putting the floor under brcount does not change the output in any way, just remove it. Change the termination test from ==0 to <=0 to prevent infinite loops in case somebody does something truly wacko in the code. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-27docs: kdoc: don't reinvent string.strip()Jonathan Corbet
process_proto_type() and process_proto_function() reinventing the strip() string method with a whole series of separate regexes; take all that out and just use strip(). The previous implementation also (in process_proto_type()) removed C++ comments *after* the above dance, leaving trailing whitespace in that case; now we do the stripping afterward. This results in exactly one output change: the removal of a spurious space in the definition of BACKLIGHT_POWER_REDUCED - see https://docs.kernel.org/gpu/backlight.html#c.backlight_properties. I note that we are putting semicolons after #define lines that really shouldn't be there - a task for another day. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-27docs: kdoc: split the processing of the two remaining inline statesJonathan Corbet
Now that "inline_*" are just ordinary parser states, split them into two separate functions, getting rid of some nested conditional logic. The original process_inline() would simply ignore lines that didn't match any of the regexes (those lacking the initial " * " marker). I have preserved that behavior, but we should perhaps emit a warning instead. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-27docs: kdoc: remove the inline states-within-a-stateJonathan Corbet
The processing of inline kerneldoc comments is a state like the rest, but it was implemented as a set of separate substates. Just remove the substate logic and make the inline states normal ones like the rest. INLINE_ERROR was never actually used for anything, so just take it out. No changes to the generated output. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-26docs: kdoc: remove the INLINE_END stateJonathan Corbet
It is never used, so just get rid of it. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-25docs: kdoc: rework process_export() slightlyJonathan Corbet
Reorganize process_export() to eliminate duplicated code, don't look for exports in states where we don't expect them, and don't bother with normal state-machine processing if an export declaration has been found. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-25docs: kdoc: remove KernelEntry::functionJonathan Corbet
This member is unused, to take it out. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-25docs: kdoc: remove a bit of dead codeJonathan Corbet
The type_param regex matches "@..." just fine, so the special-case branch for that in dump_section() is never executed. Just remove it. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-25docs: kdoc: Move content handling into KernelEntryJonathan Corbet
Rather than having other code mucking around with this bit of internal state, encapsulate it internally. Accumulate the description as a list of strings, joining them at the end, which is a more efficient way of building the text. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-25docs: kdoc: remove KernelEntry::in_doc_sectJonathan Corbet
This field is not used for anything, just get rid of it. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-25docs: kdoc: finish disentangling the BODY and SPECIAL_SECTION statesJonathan Corbet
Move the last SPECIAL_SECTION special case into the proper handler function, getting rid of more if/then/else logic. The leading-space tracking was tightened up a bit in the move. Add some comments describing what is going on. No changes to the generated output. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250621203512.223189-10-corbet@lwn.net
2025-06-25docs: kdoc: Add some comments to process_decl()Jonathan Corbet
Now that the function can actually fit into a human brain, add a few comments. While I was at it, I switched to the trim_whitespace() helper rather than open-coding it. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250621203512.223189-9-corbet@lwn.net
2025-06-25docs: kdoc: coalesce the end-of-comment processingJonathan Corbet
Separate out the end-of-comment logic into its own helper and remove the duplicated code introduced earlier. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250621203512.223189-8-corbet@lwn.net